DOCUMENT RESUME 



ED 364 568 



TM 020 721 



AUTHOR 
TITLE 



INSTITUTION 
SPONS AGENCY 

REPORT NO 
PUB DATE 
CONTRACT 
NOTE 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Christal, Raymond E. 

Comparative Validities of ASVAB and LAMP Tests for 

Logic Gates Learning. Final Technical Paper for 

Period November 1989-March 1991. 

Universal Energy Systems, Dayton, OH* 

Armstrong Lab, Brooks AFB, TX. Human Resources 

Directorate. 

AL-TP-1991-0031 

Jun 91 

F41689-86-D-0052 
81p. 

Reports - Research/Technical (143) 
MF01/PC04 Plus Postage. 

Adults; * Aptitude Tests; Cognitive Tests; Comparative 
Testing; Computer Assisted Testing; Individual 
Differences; Knowledge Level; Logical Thinking; 
^Military Personnel; Occupational Tests; ^Predictive 
Measurement; Problem Solving; *Skill Development; 
Test Use; *Test Validity 

Air Force; *Armed Services Vocational Aptitude 
Battery; Knowledge Acquisition; ^Learning Abilities 
Measurement Program; Logic Gates; Procedural Skills; 
Variance (Statistical) 



ABSTRACT 

Two courses on logic gates were administered to 1,191 
male and female recruits in Basic Military Training in 3 studies 
(448, 431, and 312 subjects, respectively) to evaluate the relative 
validities of the Armed Services Vocational Aptitude Battery (ASVAB) 
and the Learning Abilities Measurement Program (LAMP) tests in 
predicting individual differences in the acquisition of declarative 
knowledge (measured by the accuracy in solving problems) and the 
development of procedural skill (measured by the time required to 
solve problems). In general, the LAMP tests added about 20 percent 
unique valid variance to the ASVAB tests in predicting these 
criteria. Furthermore, for both criteria, the LAMP test alone 
accounted for more variance than did the ASVAB tests alone. Analyses 
indicate that the advantage of the LAMP tests was neither due to 
concurrency effects, nor obtained at the expense of excessive testing 
time. Although LAMP tests and the criterion tests used common 
methods, arguments are presented contending that most of the 
covariance associated with common methods was meaningful to the task 
being learned and performed. Thirty tables in the text and six more 
in two appendixes present study findings. (Contains 22 references.) 
(Author/SLD) 



* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 



QO 

so 
m 

a 



A 
R 
M 
S 
T 
R 
O 
N 
G 



AL-TP-1991-0031 



U.S. DEPARTMENT OF EDUCATION 
Otftce of Educational Reseaicn and imp«>tm€ni 
EDUjCATlONAL RESOURCES INFORMATION' 

/ CENTER (ERlO 

j/This document has oeen reproduced a* 

receded irom trse person or organization 

originating >t 
C Minor cnanges nave been made to .mpiove 

reO'Oduction Quality 

* Pomta ot v«ew Of op.mons siaied «n tn.s docu- 
ment do not nece*M'«iv represent oM.cai 
OERI position or policy 




COMPARATIVE VALIDITIES OF ASVAB AND LAMP 
TESTS FOR LOGIC GATES LEARNING 



Raymond E. Christa! 

Universal Energy Systems 
4401 Dayton-Xenia Road 
Dayton, OH 45432-1894 



L 
A 
B 
O 
R 
A 
T 
O 
R 
Y 



HUMAN RESOURCES DIRECTORATE 
MANPOWER AND PERSONNEL RESEARCH DIVISION 
Brooks Air Force Base, TX 78235-5000 



June 1991 

Final Technicel Paper for Period November 1989 - March 1991 



Approved for public release; distribution is unlimited. 




AIR FORCE SYSTEMS COMMAND 
BROOKS AIR FORCE BASE, TEXAS 78235-5000 



BEST COPY AVAILABLE 



NOTICES 



This technical paper is published as received and has not been edited by the 
technical editing staff of the Armstrong Laboratory. 

When Government drawings, specifications, or other data are used for any 
purpose other than in connection with a definitely Government-related 
procurement, the United States Government incurs no responsibility or any 
obligation whatsoever The fact that the Government may have formulated or in 
any way supplied the said drawings, specifications, or other data, is not to be 
regarded by implication, or otherwise in any manner construed, as licensing the 
holder, or any other person or corporation; or as conveying any rights or permission 
to manufacture, use, or sell any patented invention that may in anyway be related 
thereto. 

The Office of Public Affairs has reviewed this paper, and it is releasable to the 
National Technical Information Service, where it will be available to the general 
public, including foreign nationals. 

This paper has been reviewed and is approved for publication. 




SCOTT R. CHAIKEN 
Contract Monitor 





fonel, USAF 
tesearch Division 



BEST COPY AVAILABLE 



3 



REPORT DOCUMENTATION PAGE 


Form Approved 
0MB NO. 0704-0188 


Public reporting burd«n for this collection of information is tstlmatsd to avsrags 1 hour par ratpons*. including ths tin* for rsvtowing instructions, ••arching sxistlng data tourcss. 
gathering and maintaining ths data nasdad. and completing and r«vi«Mng tha collection of Information. Sendcommems regarding This burden estimate or any other aspect of this 
collection of Information, including suggestions for reducing this burden, To Washington Headquarters Services. Directorate for information Operations and Reports, 1215 Jefferson 
Daws Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-O1M), Washington, DC 20603. 


1. AGENCY USE ONLY (l«av« blank) 


2. REPORT DATE 

June 1991 


3, REPORT TYP 

Interim Tec 


E AND DATES COVERED 

h Paper -Nov 1989- Mar 1991 



4. TITLE AND SUBTITLE 

Comparative Validities of ASVAB and LAMP Tests for Logic Gates Learning 



6. AUTHOH(S) 

Raymond E. Christal 



5. FUNDING NUMBERS 

C - F41689-86-D-0052 
PE - 61102F 
PR - 2313 
TA - T1 
WU - 33 



PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

Universal Energy Systems 
4401 Dayton-Xenia Road 
Dayton, OH 45432-1894 



8. PERFORMING ORGANIZATION 
REPORT NUMBER 



9. SPONSORING/MONITORING AGENCY NAMES(S) AND ADDRESS(ES) 

Armstrong Laboratory 
Human Resources Directorate 
Manpower and Personnel Research Division 
Brooks Air Force Base, TX 78235-5000 



10. SPONSORING/MONITORING AGENCY 
REPORT NUMBER 

AL-TP-1 991 -0031 



11. SUPPLEMENTARY NOTES 

Armstrong Laboratory Technical Monitor: Scott R. Chaiken, Ph.D. (512) 536-3713. 



12a. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution is unlimited. 



12b. DISTRIBUTION CODE 



13. ABSTRACT (Maximum 200 words) 

Two courses on logic gates were administered to samples of airmen in Basic Military Training to evaluate the 
relative validities of ASVAB and LAMP tests in predicting individual differences in the acquisition of declarative 
knowledge (measured by the accuracy in solving problems) and the development of procedural skill (measured by 
the time required to solve problems). In general, the LAMP tests added about 20% unique valid variance to the 
ASVAB tests in predicting these criteria. Furthermore, for both criteria, the LAMP tests alone accounted for more 
variance than did the ASVAB tests alone. Analyses indicated that the advantage of the LAMP tests was not due to 
concurrency effects, nor was it obtained at the expense of excessive testing time. Although LAMP tests and the 
criterion tests used common methods, arguments are presented contending that most of the covariance associated 
with common methods was meaningful to the task being learned and performed. 



14. SUBJECT TERMS 

ASVAB 
cognition 
cognitive ability 
computerized testing 



individual differences validity 
learning 

learning abilities measurement program (LAMP) 
learning ability 



15. NUMBER OF PAGES 

82 



16. PRICE CODE 



17. SECURITY CLASSIFICATION 
OF REPORT 

Unclassified 



1S. SECURITY CLASSIFICATION 
OF THIS PAGE 

Unclassified 



19. SECURITY CLASSIFICATION 
OF ABSTRACT 

Unclassified 



20. LIMITATION OF ABSTRACT 

UL 



ERLC 



NSN 73404 1-2*0-6800 



Standard Form 296 fffev. 2*88) 
Praicribtd by ANSI Sd. Z0*\t 
2M-T02 



TABLE OF CONTENTS 

Page 

SUMMARY 1 

INTRODUCTION 2 

GENERAL APPROACH 3 

METHOD 4 

Subjects 4 

Testing Facilities 5 

Procedure 5 

STUDY 1 10 

Learning Performance 10 

Factor Analysis... 12 

Correlation of Factor Scores with Learning Criteria 13 

Accuracy Scores * 13 

Latency Scores 14 

Comparative Validities of ASVAB and LAMP Tests for Gates-Course Learning Criteria 16 

Acquisition of Declarative Knowledge 16 

Acquisition of Procedural Skill 19 

STUDY 2 25 

Factor Analysis 26 

Comparative Validities of ASVAB and LAMP Tests for Gates-Course Learning Criteria 29 

Acquisition of Declarative Knowledge 29 

Acquisition of Procedural Skill 31 

Discussion 34 

Decision on Need for Study 3 36 

STUDY 3 36 

CORRECTIONS FOR RESTRICTIONS OF RANGE .'. 39 

Description of the Problem 39 

Approach 39 

Results 40 

GENERAL DISCUSSION 48 

Nature of the Learning Task 49 

Concurrency Effects 49 

Common Methods 50 

Processing Accuracy 51 

Testing Time 51 

Other Considerations 52 

CONCLUSIONS 53 

REFERENCES 54 

APPENDIX A - Descriptions of ASVAB and LAMP Tests 57 

APPENDIX B«Factor Analyses of Study 1 and Study 2 Test Scores 64 

APPENDIX C- Validities of ASVAB and LAMP Tests for Study 1 and Study 2 Accuracy 

and Latency Criteria 68 

LIST OF FIGURES 

Figure Page 

1 Test Abbreviations, Titles, and Study Usuage 8 

2 Codes, Definitions, and Study Usuage for Logic Gates Course Blocks 9 



LIST OF TABLES 



Table Page 

1 Means and Standard Deviations for Study 1 Course Blocks and Averages 10 

2 Intercorrelations of Accuracy Scores from Study 1 Course Blocks 11 

3 Intercorrelations of Latency Scores from Study 1 Course Blocks 12 

4 Correlations of Factor Scores with Study 1 Course Accuracy Criteria 14- 

5 Correlations of Factor Scores with Study 1 Course Latency Criteria 15 

6 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 1 Accuracy 

Criteria, Estimated by Approach 1 17 

7 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 1 Accuracy 

Criteria, Estimated by Approach 2 19 

8 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 1 Latency 

Criteria, Estimated by Approach 1 20 

9 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 1 Latency 

Criteria, Estimated by Approach 2 21 

10 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 1 Latency 

Criteria, Estimated by Approach 1 in the Residual Sample (N=344) 23 

11 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 1 Latency 

Criteria, Estimated by Approach 2 in the Residual Sample (N=344) 24 

12 Means and Standard Deviations for Study 2 Blocks and Averages 26 

13 Correlations of Factor Scores with Study 2 Accuracy Criteria 27 

14 Correlations of Factor Scores with Study 2 Latency Criteria 28 

15 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 2 Accuracy 

Criteria, Estimated by Approach 1 29 

16 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 2 Accuracy 

Criteria, Estimated by Approach 2 30 

17 Unique Contributions of ASVAB and LAMP Tests to the Predicition of Study 2 Latency 

Criteria, Estimated by Approach 1 32 

18 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 2 Latency 

Criteria, Estimated by Approach 2 33 

19 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 2 Latency 

Criteria, Estimated by Approach 1 in the Residual Sample (N=368) 34 

20 Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 2 Latency 

Criteria, Estimated by Approach 2 in the Residual Sample (N=368) 35 

21 Comparative Multiple Correlations of the Entire ASVAB and a Subset of ASVAB Tests 

for Accuracy Criteria in Study 1 and Study 2 37 

22 Predictive and Concurrent Multiple Correlations of a Subset of ASVAB tests for Study 2 

Accuracy Criteria 38 

23 Validities of ASVAB and LAMP Tests for Study 1 Accuracy and Latency Criteria in the 

Computing Sample, the Air Force Applicant Sample, and the 1980 Youth Sample 41 

24 Validites of ASVAB and LAMP Tests for Study 2 Accuracy and Latency Criteria in the 

Computing Sample, the Air Force Applicant Sample, and the 1980 Youth Sample 42 



ERLC 



IV 



25 Contribution of LAMP Tests to ASVAB Tests in Accounting for Study 1 Criterion Variance 



in the Air Force Applicant Sample 43 

26 Contribution of LAMP Tests in Accounting for Study 1 Criterion Variance in the 

1980 Youth Sample 44 

27 Contribution of LAMP Tests in Accounting for Study 2 Criterion Variance in the 

Air Force Applicant Sample « 45 

28 Contribution of LAMP Tests in Accounting for Study 2 Criterion Variance in the 

1980 Youth Sample 46 

29 Summary of LAMP Test Contributions in Accounting for Study 1 and Study 2 

Acuracy and Latency Criterion Variance 47 

30 Summary of Increases in Efficiency Contributed by LAMP tests to ASVAB Tests in the 

Prediction of Accuracy and Latency Criteria 48 

Bl Results from Factor Analysis of Study 1 Predictor Tests 64 

B2 Results from Factor Analysis of Study 2 Predictor Tests 67 

CI Validities of ASVAB and LAMP Tests for Study 1 Accuracy Criteria 68 

C2 Validities of ASVAB and LAMP Tests for Study 1 Latency Criteria 69 

C3 Validities of ASVAB and LAMP Tests for Study 2 Accuracy Criteria 70 

C4 Validities of ASVAB and LAMP Tests for Study 2 Latency Criteria 71 



ERLC 



7 

V 



PREFACE 



The conduct of this research and the preparation of this report was supported under a contract 
designed to service the Air Force Learning Abilities Measurement Program (LAMP), a multi-year 
program of basic research conducted at the U.S. Air Force Human Resources Laboratory (AFHRL)* , 
and sponsored by the Air Force Office of Scientific Research (AFOSR). The goals of this program are 
to specify the basic parameters of learning ability, to develop techniques for the assessment of 
individuals' knowledge and skill levels, and to explore the feasibility of a model-based system of 
psychological assessment. The present study reports on applied, rather than basic research, and was 
conducted to help determine whether the new types of computer measures coming out of LAMP 
research show promise for ultimately improving the Air Force personnel selection and classification 
system. The author wishes to thank Scott Chaiken, Patrick Kyllonen, Dan Woltz, Bill Alley, Valerie 
Shute, and Bob Young for their review of the draft manuscript and their helpful comments. Thanks 
are also given to Bill Tirre and Malcolm Ree for their assistance in obtaining data and information 
used in correcting validities and intercorrelations for restrictions in range due to selection. Finally, 
special appreciation is expressed to Terri Purdue who both programmed the logic gates courses and 
prepared all data for the statistical analyses, and to Janice Hereford, who programmed data for the 
restriction of range corrections. 



ERLC 



* AFHRL has been redesignated Human Resources Directorate, Armstrong Laboratory. 



vi 

8 



COMPARATIVE VALIDITIES OF ASVAB AND LAMP TESTS FOR LOGIC 

GATES LEARNING 

SUMMARY 

Two short computer courses were designed to teach basic airmen how to identify various types 
of logic gates and to determine their output signals as a function of their input signals. In Study 1, 448 
subjects received training on simple gates with and without negations. After training, they were 
required to solve six 36-item blocks of gate problems without negations, and four 36-item blocks of gate 
problems with negations. Instructive feedback was provided after each incorrect response. In Study 2, 
431 subjects were trained on simple gates and required to solve eight 36-item blocks of simple gate 
problems without negations, and one 30-item block of simple gate problems with negations. Instructive 
feedback was given after each incorrect response. These blocks were followed by two 60-item blocks of 
mixed simple gates (with and without negations), training on complex gate circuits, and, finally, four 54- 
item blocks of mixed simple gates and complex gate circuits. 

Subject responses were scored both for accuracy and for latency. Accuracy scores were used 
as criteria indicating the acquisition of declarative knowledge concerning the rules for solving logic gate 
problems. Latency scores at each stage were used as criteria indicating the level of procedural skill 
acquired up to that stage. In each study, the subjects were administered LAMP tests designed to 
measure working memory capacity and information processing speed. ASVAB test scores were 
obtained for subjects from operational files. 

One of the primary purposes of the study was to determine the relative power of the the 
ASVAB tests and of the LAMP tests in predicting the learning criteria from the logic gate courses. It 
was also considered important to determine how much unique valid variance the LAMP tests could 
contribute to the ASVAB tests in predicting these criteria. In general, the LAMP tests were found to 
add about 20% unique valid variance to the ASVAB in predicting both accuracy and latency block 
averages in the two courses. When the intercorrelations and validities were corrected for restrictions of 
range due to selection, the estimated contributions of LAMP tests to ASVAB tests in accounting for 
criterion variance ranged from 17% - 21% in the U.S Air Force (USAF) applicant sample, and from 
13% - 20% in the 1980 youth san» k le. Furthermore, against all criteria, LAMP tests alone accounted 
for more variance than did ASVAB alone. Analyses were conducted in a third study which indicated 
that the advantage of the LAMP tests over the ASVAB tests was not the result of a concurrency effect, 
nor was it obtained at the expense of excessive testing time. The results are viewed as encouraging 
with respect to the power of LAMP tests to predict meaningful learning criteria. 



INTRODUCTION 



The Learning Abilities Measurement Program (LAMP) is a basic research program jointly 
sponsored by the Air Force Office of Scientific Research (AFOSR) and the Air Force Human 
Resources Laboratory (AFHRL). It supports studies on the identification and measurement of 
abilities which can account for individual differences in the acquisition of knowledge and skills. The 
present framework for the LAMP program is loosely modeled after Anderson's (1983) ACT* theory, 
and hypothesizes that individual differences in learning efficiency arise from four primary sources: 
a) the breadth and depth of declarative knowledge brought to the learning situation; b) the breadth 
and depth of procedural skills brought to the learning situation; c) information processing speed 
capabilities; and d) working memory capacity (Kyllonen, 1985b). Research during the past several 
years has been focused on defining, measuring and validating abilities relating to processing speed and 
working memory capacity (Kyllonen,1985a; Kyllonen & Christal, 1989; Kyllonen & Christal, 1990; 
Kyllonen, Tirre, & Christal, 1988; Woltz, 1988). 

Although LAMP is a basic research program, it is nevertheless goal oriented. The ultimate 
goal is to improve the U.S. Air Force (USAF) personnel selection and classification system. In order 
to gauge whether progress is being made toward this goal, it is desirable to periodically validate new 
tests against meaningful learning criteria. Ultimately, there will be a large-scale validation against 
school and job performance criteria. However, it may be some time before the results of basic 
research will warrant this type of costly validation effort. In the meantime, in order to generate 
meaningful learning criteria for test validation, the approach has been to train individuals to proficiency 
on operational tasks which are brought into the laboratory. A new experimental training facility, the 
Complex Learning Assessment (CLASS) laboratory, has been established for this purpose. Individuals 
participating in CLASS experiments spend approximately 1 to 1 1/2 days taking experimental LAMP 
tests, and then up to 30 hours taking a training course by means of intelligent tutors administered by 
microcomputers. 1 Rich information concerning skill and knowledge acquisition is recorded during 
course administration, which, in turn, is used to investigate learning behaviors and to validate scores 
from the concurrently administered experimental LAMP tests. So far, data have been collected from 
two such courses, one in computer programming and one in electrical circuits. A third course, dealing 
with flight-engineering tasks, is being constructed (Shute, in press-a; Shute, in press-b; Shute & Glaser, 
1990, Shute & Pena , 1990). 



l There are wide individual differences in both the time required to take the experimental tests and to 
complete the training courses. 
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Although data from CLASS experiments are extremely useful, developing and administering 
intelligent tutors is a somewhat slow and expensive process. It requires more than a year to develop 
and administer a course. To supplement this effort, attempts have been made recently to develop short 
(1- to 2-hour) courses, on narrow subjects, which can be administered to samples of airmen in Basic 
Military Training. The LAMP project has access to samples of basic airmen for approximately 3 hours 
of experimental testing during their 11th training day. This paper describes the development of courses 
on electronic logic gates, which were used in three studies to validate LAMP tests and tests from the 
Armed Services Vocational Aptitude Battery (ASVAB). 

Logic gates were chosen for the course subject-matter for several reasons. First, they have 
been used by other investigators to study skill acquisition (Carlson, Khoo, Yaure & Schneider, 1990). 
Second, they lend themselves to the development of test items at various difficulty levels. 2 Third, gate 
problems were predicted to be well suited for studies on procedural skill development. Finally, and 
this is the most important reason, previous research has demonstrated that knowledge of logic gates is 
a distinguishing factor between "skilled and less-skilled" lst-term airmen working in avionics 
(Gitomer,1984). In light of these observations, it was judged that a short course on logic gates would 
fill the requirement for a meaningful learning task which could be administered under laboratory 
conditions in less than 2 hours. 



The present paper reports the results of three studies. The first half of Study 1 involved 
training on AND, OR, and XOR gates, followed by six blocks of test questions on simple gates without 
negations. In the second half of Study 1 course instructions were provided on the effects of negations, 
and included four blocks of questions on simple gates with negations. In both sections, instructive 
feedback was provided after each incorrect response. 3 



2 Simple AND or OR gates without negations are easy. XOR gates are somewhat more difficult. Gates 
with negations are considerably more difficult to learn than are gates without negations. Complex 
circuits involving several gates can be very difficult to solve. 

3 A distinction is made between testing blocks and analysis blocks. Study 1 included three 72-item 
testing blocks of simple gates without negations and two 72-item testing blocks of simple gates with 
negations. For analysis purposes, these testing blocks were split inlo six and four 36-item blocks, 
respectively. Study 2 included four 72-item blocks of simple gates without negations, which were split 
into eight 36-item blocks for the analysis. In all other instances, testing blocks were identical to analysis 
blocks. Summary feedback was provided to subjects after each testing block. 



GENERAL APPROACH 
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In Study 2, subjects were given a pre-test to determine their level of "going-in" knowledge. 
This was followed by: (a) instructions on simple AND, OR, and XOR gates; (b) eight blocks of test 
questions on simple gates without negations, with instructive feedback after incorrect responses; (c) 
instructions on negations; (d) one block of simple gates with negations, with instructive feedback after 
incorrect responses; (e) two blocks of mixed simple gates, with and without negations; (0 instructions 
on complex gate circuits; and (g) three blocks of test questions on mixed simple and complex gates, 
some with and some without negations. 

In both Study 1 and Study 2, subjects were administered experimental tests developed in the 
LAMP program. Scores from the ASVAB were obtained for each subject from operational files. A 
series of analyses was conducted to determined how well individual differences in the acquisition of 
declarative knowledge (reflected in accuracy scores) could be predicted by LAMP tests alone and by 
ASVAB tests alone. Analyses were also conducted to see how much predictive validity the LAMP tests 
added to ASVAB tests in the prediction of gates course criteria. Similar analyses were conducted to 
determine the power of the LAMP and ASVAB tests to predict the development of procedural skill, 
which was reflected in the time required to solve gate problems. 

One of the concerns in Studies 1 and 2 was that the LAMP tests were administered 
concurrently (in the same test session) with the gates course, while the ASVAB tests had been 
administered several months earlier in operational testing stations. Previous research has suggested 
that concurrent administration of predictor and criterion measures may lead to a minor inflation in 
estimated validity coefficients (Christal, 1989; Divgi, 1990). Study 3 was conducted to evaluate the 
magnitude of this concurrency effect. It involved the re-admirlstration of several ASVAB tests with 
the Study 2 gates course. This made possible a comparison of predictive (non-concurrent) and 
concurrent validity coefficients for the same criteria. 

More detailed descriptions of the gates courses are given in the sections which follow, and 
abbreviated descriptions of the LAMP and ASVAB tests are provided in Appendix A. 
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METHOD 

Subjects 

For Study 1, the subjects were 448 military recruits who were tested on the 6th day of their 
basic military training at Lackland Air Force Base, Texas. Of these, 94% were males and 6% females. 
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Studies 2 and 3, included 431 and 312 subjects, respectively, who * tested on the 11th day of then- 
basic military training. In Study 2, 7.6% of the subjects were females, while in Study 3, 28.2% were 
females. The larger proportion of females in Study 3 was fortuitous. Testing samples were drawn from 
training flights as they arrived, and such flights were either all-male or all-female. Not included in the 
above samples were subjects for whom ASVAB scores could not be obtained, those who did not 
complete the course and experimental tests in the available testing time, and up to 1% who were 
eliminated because they were extreme outliers on the course on one or more of the tests. 



Testing Facility 

The testing facility consisted of partitioned testing stations in a large air-conditioned room. 
Each station was equipped with a Zenith Z-248 microcomputer with a 20 Mb hard disk for storing 
course materials, test items, and subject responses. A standard keyboard was used for response entry, 
and an enhanced graphics adaptor (EGA) and display monitor (640 x 350 color resolution) was used 
for stimulus presentation. 

Procedure 

For each testing session, a proctor briefed subjects on the purposes of the testing, and then 
assigned them to testing stations. A computer program provided subjects with general information 
about the LAMP program; described use of the keyboard; told how to ask questions of the proctor; 
provided information about breaks; warned against talking with others about the test during breaks; 
and indicated what to do when the course and all tests were completed 

The subjects read and responded to information in accordance with the Privacy Act. They 
entered their Social Security Number, which was required for use in acquiring their ASVAB scores, 
and answered questions concerning their gender, education, and race. Then they responded to a short 
questionnaire concerning the use of English as a first language at home and in school. 

In each of the three studies, the subjects were given a general description of logic gates, and 
answered 12 questions concerning whether they had studied logic gates in school, had read about them 
in books, magazines, or newspapers, knew how they operate, or had used them in building electronic 
projects. They also answered questions concerning prior experience with microcomputers, interest in 
electronics and science, and whether they had sought an assignment in a USAF electronics career 
ladder. 
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In Studies 2 and 3, the above questionnaire was administered immediately before or 
immediately after a pre-test designed to measure subjects' levels of incoming knowledge specifically 
relating to logic gates. These gate problems were taktn directly from the course which was to follow, 
except that additional foils (distractors) were generated to attract individuals who had little or no 
knowledge concerning logic gates. 

In all three studies, the subjects were provided with a short training course on the shapes of 
AND, OR, and XOR gates, and were asked to associate the names with their respective shapes. This 
gate familiarization was followed with a 30-item test in which gates were presented on the screen and 
the subjects were asked to indicate the associated name by pressing an appropriate key. After each 
incorrect response, the subjects were informed of their incorrect response and were presented, for 10 
seconds, with a screen which displayed all three gates and their associated names. In all three studies, 
the first two sections of the course were identical, except for the number of test problems and the 
method used for timing instruction frames. Each successive screen was displayed for a specific period, 
and then the next screen automatically appeared. The keyboard was iocked-out', except during the 
times the subject was required to enter responses to problems. In Study 1, the direction screens 
contained statements concerning time allowances (e.g. "You will have 45 seconds to study information 
on this screen."). In Studies 2 and 3, these time allowance statements were replaced by timing bars at 
the top of the screen, which became shorter with the passing of each second. Otherwise, the directions 
for the first two sections were identical in all three studies, and can be summarized as follows: 

(1) A 45-second screen describing logic gates and displaying a gate drawing with labeled input 
leads, gate body and output lead. 

(2) A 45-second screen describing the input-output rules for the AND gate, with two examples 
provided. 



(3) A screen providing six labeled examples of AND gates, some with two and some with three 



and 3, these examples were added to a screen one-at-a-time, after specified periods of time. In all 
studies, the correct output signals (problem answers) were provided. 

(4) The subjects were given a test containing eight AND gates, in which they indicated whether 
the output signal was HI or LO by pressing the 'H' or key. After each incorrect response, the 
subject was shown a screen for ten seconds which presented rules for the particular gate-type in 
question. 



input leads. In Study 1, these examples were presented all at once on a 45-second screen. In Studies 2 
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(5) The subjects were shown a series of instruction screens for OR gates. This was followed 
by an 8-problem test on OR gates, with a 10-second rule-feedback for each item missed. 

(6) The subjects were shown a series of instruction screens for XOR gates. This was followed 
by an 8-problem test on XOR gates, with a 10-second rule-feedback for each item missed. 

(7) The rules for AND, OR, and X&R gates were repeated. 

(8) In Study 1, the subjects solved six 36-item blocks of mixed AND, OR, and XOR gates, 
without negations. After each incorrect response, the subject was provided with a 10-second instructive 
feedback frame which displayed the pictures and rules for all three gate-types, with the missed gate- 
type highlighted. Studies 2 and 3 differed only in the fact that they included eight blocks (as opposed to 
six in Study 1), and the feedback screens contained timing bars. In all studies, summary information 
concerning percent correct and median response latency was given at the end of each testing block. 

(9) Instead of complicating the study with definitions of NAND, NOR gates and 
INVERTERS, the present studies made use of negation symbols on input and output leads. The 
subjects were first taught the impact of a negation on the output signals of AND, OR, and XOR gates, 
with six examples provided. Then the impact of negations on input leads was explained, again with six 
examples given. Finally, an example was given with negations on both input and output leads. As in 
the case of simple gates without negations, the study frames were presented for specific time periods, 
with timing bars used in Studies 2 and 3. In Study 1, the subjects were administered four problem 
blocks, in which all items contained negations on the input leads, output leads, or both input and output 
leads. Latency feedback was given for correct responses, and instructive feedback was given for 
incorrect responses. Summary feedback on percent correct and median latency was provided after 
each testing block. In Studies 2 and 3, only one 30-item test block of gates with negations was given. 

In Studies 2 and 3, the one block of simple gates with negations was followed by two 60-item 
blocks containing a mixture of simple gates with and without negations. Then the subjects were given 
instructions on how to determine the final output signal of complex circuits involving two input gates, 
with or without negations, and one output lead with or without negations. Finally three 54-problem 
blocks of were presented, each of which contained a mixture of simple gates (with and without 
negations), and complex gates (with and without negations). Feedback in the form of 
CORRECT/WRONG and response latency was provided for each of the items in the mixed simple 
gates and the mixed simple/complex gates blocks, along with summary data at the end of each testing 
block. Although details will not be provided, the blocks within sections were balanced in terms of item 
types. 

o 7 15 

ERIC 



Figure 1 presents the abbreviations for the predictor tests, and Figure 2 presents abbreviations 
for the course criteria. Both sets of abbreviations are used in tables throughout the remaining part of 
this paper. 

Figure L Test Abbreviations, Titles and Study Usage 
Abbreviation Title Studyl Study2 Study3 



LAMP 



GS 


General Science 


A. 


Y 




AK 


Arithmetic Reasoning 




x 

yv 


x 


AKK 


/vriLtuneuc Reasoning, Keiesi 






x 


XI TV 

WK 


Word Knowledge 


Y 

y\. 


X 
yv 




rt 


Paragraph Comprehension 


X 
yv 


X 

yv 




ISCJ 


Numerical Operations 


X 
y\. 


X 

yv 




(Jo 


cooing opeea 


X 


X 

yv 




AS 


Auto and Shop Information 


V 

A. 


V 
yv 




MK 


Mathematics Knowledge 


.A. 


yv 


Y 

/\ 


MKK 


Mathematics Knowledge, Retest 






X 


ML 


Mecnamcai i^omprenension 


X 


X 
yv 


X 


MCK 


Mechanical Comprehension, Retest 






X 


CI 


cicciricai imurmaLiun 


X 


x 






VvIlUHC xVCadlULL, /A.HU1 CiL-j 


X 


x 




PPT 


V^nOlCC IxCatUUIlf JUaLCnCjr 


X 


x 




JTlrV 


jt nybicai lucnuiy, r\ccuiacjr 


X 


x 




DTT 


rnysicai identity, Latency 


X 

yv 


x 




NIA 


Name Identity, Accuracy 


X 


X 




NIL 


Name Identity, Latency 


X 


X 




MIA 


Meaning Identity, Accuracy 


X 






MIL 


Meaning Identity, Latency 


X 






NFA 


Number Fact Retrieval, Accuracy 


X 






NFL 


Number Fact Retrieval, Latency 


X 






ABA 


Grammatical Reasoning, Accuracy 


X 


X 




ABL 


Grammatical Reasoning, Latency 


X 


X 




NRA 


Number Reduction, Accuracy 


X 






DSA 


Digit Span, Accuracy 


X 






NAA 


ABC Numerical Assignment, Accuracy 


X 


X 




GKA 


General Knowledge, Accuracy 




X 




GKL 


General Knowledge, Latency 




X 




WAA 


Word Association, Accuracy 




X 




FRA 


Figure Recognition, Accuracy 




X 




MCA 


Mental Character Generation, Accuracy 




X 





Notes: See Appendix A for test descriptions. 
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Figure 2. Codes, Definitions, and Study Usage for Logic Gates Course Blocks 

Code Description Studyl Study2 Study3 



1-1 

X-X 




36 


36 


36 


1-2 


Simple Gates, No Negations, Block 2 


36 


36 


36 


1-3 


Simple Gates, No Negations, Block 3 


36 


36 


36 


1-4 


Simple Gates, No Negations, Block 4 


36 


36 


36 


1-5 


Simple Gates, No Negations, Block 5 


36 


36 


36 


1-6 


Simple Gates, No Negations, Block 6 


36 


36 


36 


Avl 


Simple Gates, No Negations, Average, Blks 1-6 


216 






1-7 


Simple Gates, No Negations, Block 7 




36 


36 


1-8 


Simple Gates, No Negations, Block 8 




36 


36 


Avl 


Simple Gates, No Negations, Average, Blks 1-8 




288 


288 


2-1 


Simple Gates, With Negations, Block 1 


36 


30 


30 


2-2 


Simple Gates, With Negations, Block 2 


36 






2-3 


Simple Gates, With Negations, Block 3 


36 






2-4 


Simple Gates, With Negations, Block 4 


36 






Av2 


Simple Gates, With Negations, Blks 1-4 


144 







3-1 Mixed Simple Gates, With & Without Negations, Block 1 60 60 

3-2 Mixed Simple Gates, With & Without Negations, Block 2 60 60 

Av3 Mixed Simple Gates, With & Without Negations, Blks 1-2 120 120 



4-1 Mixed Simple & Complex Gates, With & Without Neg. Blk 1 54 54 

4-2 Mixed Simple & Complex Gates, With & Without Neg. Blk 2 54 54 

4-3 Mixed Simple & Complex Gates, With & Without Neg. Blk 3 54 54 

Av4 Mixed Simple & Complex Gates, With <fe Without Neg. Avg. 1-3 162 162 

AVG Average Across All Course Blocks 360 600 600 

Notes: Codes are used in tables throughout the paper to identify blocks of accuracy and latency scores. 
The Avl values are averages across six blocks in Study 1, and across eight blocks in Study 2. Table 
values indicate the number of items in the various blocks and block averages. 
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STUDY 1 



Learning Performance 

Study 1 involved administration of six 36-problem blocks of simple gates without negations and 
four 36-item blocks of simple gates with negations. Table 1 presents the means and standard 
deviations for the accuracy scores and for the median latency scores for these various blocks. 

Table 1. Means and Standard Deviations for Study 1 Course Blocks and Averages 

BLK CODE 1-1 1-2 1-3 1-4 1-5 1-6 Avl 2-1 2-2 2-3 2-4 Av2 AVG 

ACCURACY 

Mean 79.2 87.8 91.7 932 94.8 95.2 90.3 83.8 87.0 89.2 89.6 87.4 87.9 

S.D. 11.8 10.2 8.1 7.4 6.9 6.4 6.9 15.4 13.8 12.8 12.6 12.6 7.9 

LATENCY 

Mean 1.87 1.58 1.43 1.27 1.21 1.17 1.37 3.39 2.98 2.74 2.74 2.95 1.80 

S.D. .53 .43 .39 .29 .29 .27 30 .87 .76 .71 .73 .70 .40 

Notes: Block codes are defined in Figure 2. 

It is clear from data in Table 1 that both declarative and procedural learning took place across 
the entire course. In the case of simple gates without negations, the mean accuracy rose from 79.2% in 
Block 1 to 95.2% in block 6, while the standard deviations dropped from 11.8% to 6.4% across these 
same blocks. Since the "going-in" knowledge about logic gates for the basic airmen is near zero (see 
Study 2), the mean accuracy level of 79.2% for problems in Block 1 must be attributed to learning 
which took place in the course prior to Block 1, and to the instructional feedback given for problems 
missed within Block 1. The remaining learning (reflected in the increase in means from 79.2% to 
95.2%) can be attributed to learning from the instructive feedback given for problems missed after 
Block 1. 



ERIC 



* 8 

10 



The group data also reveal a systematic increase in the development of procedural skill. The 
mean of the median response latencies dropped from 1.87 seconds in Block 1 to 1.17 seconds in Block 
6, and the standard deviations for latencies dropped from .53 to J27 seconds across these same Blocks. 

In the case of gates with negations, the learning rates were considerably lower, rising from 
83.8% accuracy in Block 1 to 89.6% accuracy in Block 4, with the associated standard deviations 
dropping from 15.4% to 12.6%. There was some indication of the development of procedural skill 
during the first three blocks, but reductions in latencies leveled off after Block 3. 

Table 2 reports the intercorrelatons of accuracy scores for all problem blocks. The 
intercorrelation matrix for the six blocks of gates without negations (labeled 1-1 to 1-6) was in the form 
of a pseudo simplex, the highest values being between adjacent blocks and the lowest values being 
between the blocks furthermost removed from each other. The same is true for the four blocks of gate 
problems with negations. The magnitudes of the correlations between adjacent blocks provide a rough 
indication of the stability of individual differences in performance across those blocks. The lower 
intercorrelations between the blocks of gates without negations can be partly attributed to a restriction 
in range due to a ceiling effect. In Blocks 1-6, 31.7% of the subjects made perfect scores, while another 
29.9 % missed only one item. 



Table 2. Intercorrelations of Accuracy Scores from Study 1 Course Blocks 



VAR 


1-1 


1-2 


1-3 


1-4 


1-5 


1-6 


2-1 


2-2 


2-3 2-4 


1-1 


1.00 


















1-2 


.68 


1.00 
















1-3 


.59 


.72 


1.00 














1-4 


.48 


.61 


.69 


1.00 












1-5 


.43 


.56 


.60 


.73 


1.00 










1-6 


.39 


.49 


.56 


.68 


.76 


1.00 








2-1 


.39 


.42 


.42 


.44 


.50 


.49 


1.00 






2-2 


.42 


.45 


.50 


.54 


.61 


.58 


.83 


1.00 




2-3 


.42 


.48 


.49 


.59 


.63 


.59 


.76 


.85 


1.00 


2-4 


.44 


53 


.48 


.58 


.64 


.60 


.74 


.81 


.86 1.00 



Notes. Block variable codes are defined in Figure 2. 
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Table 3 presents the intercorrelations among the latency scores for the various learning blocks. 
Again, the values in this matrix are in the form a pseudo simplex, with the latter adjacent blocks of 
gates without negations correlating in the upper .80s, 



Table 3. Intercorrelations of Latency Scores from Study 1 Course Blocks 



VAR 


1-1 


1-2 


1-3 


1-4 


1-5 


1-6 


2-1 


2-2 


2-3 


2-4 


1-1 


1.00 




















1-2 


.74 


1.00 


















1-3 


.56 


.75 


1.00 
















1-4 


.54 


.73 


.88 


1.00 














1-5 


.48 


.67 


.81 


.85 


1.00 












1-6 


.47 


.62 


.77 


.82 


.88 


1.00 










2-1 


.55 


56 


.51 


.50 


.48 


.49 


1.00 








2-2 


.51 


54 


.50 


52 


.48 


.51 


.86 


1.00 






2-3 


.43 


.46 


.46 


.48 


.44 


.46 


.73 


.80 


1.00 




2-4 


.40 


.42 


36 


38 


36 


.41 


.72 


.75 


.80 


1.00 



Notes. Block variable codes are defined in Figure 2. 



Factor Analysis 

The names and descriptions of the ability tests administered to subjects in Study 1 are 
presented in Appendix A. In the case of ASVAB tests, all scores were measures of accuracy, although 
the Numerical Operations and Coding Speed tests were administered under speeded conditions, and 
the resulting scores were based on the number of correct answers recorded during the allotted time. 
Nine tests were selected from the LAMP program for inclusion in Study 1, all of which were designed 
to be measures of information processing speed or working memory capacity. Accuracy scores were 
obtained from all of the LAMP tests, but latency scores were computed only for those processing speed 
tasks that yielded high accuracy scores. 

From the listing in Figure 1, it can be seen that a total of 25 test scores were available for the 
validation analyses (10 accuracy scores from the ASVAB, 9 accuracy scores from LAMP tests, and 6 
latency scores from LAMP tests). There was interest in determining how the common factor structure 
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of the ASVAB and LAMP tests correlated with the course criteria. Therefore six principal axis factors 
were extracted from the 25-variable intercorrelation matrix and rotated orthogonally using the Varimax 
method. The results of this factor analysis are presented in Appendix B. Six factors were identified, 
which were named (1) Processing Speed; (2) Working Memory; (3) Processing Accuracy, (4) Verbal; 
(5) Technical Knowledge; and (6) Numerical Facility/Perceptual Speed. Scores on these factors were 
correlated with accuracy and latency criteria from the logic gates course. 



Correlation of Factor Scores with Learning Criteria 
Accuracy Scores 

Factor scores were computed for all subjects and correlated with accuracy and latency scores 
from the various problem blocks in the gates course. The results of these analyses are presented in 
Tables 4 and 5. Data in Table 4 clearly indicate that individual differences in the acquisition of 
declarative knowledge were highly associated with individual differences in working memory capacity. 
This association is not surprising, since the primary requirement of problem solving was to hold a set of 
complex rules in memory while processing information concerning the input-output signals of gate 
types. Errors could have been made either through memory failures concerning rules, or through the 
inaccurate execution of processing steps required in applying the rules. The working memory factor 
(WKMEM) scores correlated in the high 30's or low 40*s with accuracy scores from the individual 
problem blocks, including both those without negations and those with negations. The reduction in the 
magnitude of relationships with the last few blocks of gate problems without negations could have been 
due to the ceiling effects mentioned previously. The Working Memory factor scores correlated .50 with 
an average accuracy score across all blocks involving gates without negations, .45 with an average 
across blocks involving gates with negations, and .53 with an average across all learning blocks. 

The Processing Accuracy (PROCACC) factor scores had the second highest relationships with 
the gate-problem accuracy scores, the correlation coefficients generally running in the mid- to upper- 
.20's for individual learning blocks, and low .30*s for block averages. The Verbal and Technical 
Knowledge factors had low but significant relationships with accuracy scores on blocks involving 
negations. 
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Table 4. Correlations of Factor Scores with Study 1 Course Accuracy Criteria 
BLOCK PROCSPD WKMEM PROACC VERBAL TECHKNOW NUM/PS 



1-1 


.07 


.42 


.24 


.12 


.04 


.13 


1-2 


.10 


.43 


.27 


.09 


.07 


.09 


1-3 


.14 


.43 


.25 


.07 


.05 


.03 


1-4 


.14 


.40 


.27 


.02 


.09 


.05 


1-5 


.10 


39 


32 


-.00 


.04 


.07 


1-6 


.07 


35 


30 


.04 


.06 


.07 


Avl 


.12 


.50 


33 


.08 


.07 


.10 


2-1 


.08 


.43 


.25 


.22 


.20 


.01 


2-2 


.05 


.42 


.28 


.17 


.17 


.00 


2-3 


.08 


.42 


.26 


.16 


.17 


-.02 


2-4 


.06 


.41 


32 


.12 


.14 


.02 


Av2 


.07 


.45 


.30 


.18 


.18 


.00 


AVG 


.11 


Si 


.35 


.15 


.16 


.06 



Memory, PROACC = Processing Accuracy; TECHKNOW = Technical Knowledge; NUM/PS = 
Numerical Facility-Perceptual Speed. 



Latency Scores 

Data in Table 5 show the relationships of factor scores with median latency scores from the 
logic gates course. The highest validities were obtained from the Processing Speed factor. Significant 
relationships were also obtained between the Working Memory factor and latency criteria for the last 
four blocks of simple gates without negations. This correlation suggests that individuals with good 
working memory capacity had sufficient strength of declarative knowledge by this stage to facilitate the 
development of procedural skill. 
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Table 5. Correlations of Factor Scores with Study 1 Course Latency Criteria 



BLOCK PROCSPD WKMEM PROACC VERBAL TECHKNQW NUM/PS 



1 1 
1-1 


OA 
us* 


.uo 


18 
.lo 


01 


(¥1 


- 11 

.jo 


1 o 
1-Z 








-.Uo 


(¥1 
.UU 


. 11 


1 1 

io 




1 A 
-.10 




(XX 


-.u<j 




1-4 


.33 


-.15 


.05 


-.04 


-.01 


-.10 


1-5 


31 


-.19 


.00 


-.04 


-.04 


-.12 


1-6 


35 


-.18 


.04 


-.02 


-.02 


-.13 


A ..1 

Avl 


34 


-.12 


.06 


-.04 


-.Ul 


1/1 


2-1 


.29 


.07 


.17 


.00 


-.01 


-.17 


2-2 


32 


.01 


.14 


.01 


.04 


-.18 


2-3 


32 


.02 


.05 


-.02 


.02 


-.18 


2-4 


.34 


.05 


.10 


.06 


.03 


-.20 


Av2 


.35 


.05 


.13 


.02 


.02 


-.19 


AVG 


35 


-.04 


.12 


-.02 


.01 


-.18 



Notes. Block codes are defined in Figure 2. 
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Significant relationships were also found between the Numerical Facility/Perceptual Speed 
factor and gates-course latency criteria. However, in a subsequent analysis, it was revealed that all of 
these relationships were accounted for by the LAMP Number Fact Retrieval test. Neither the ASVAB 
Numerical Operations test nor the Coding Speed test contributed to the prediction of latency criteria. 



Comparative Validities of ASVAB and LAMP Tests for Gates-Course Learning Criteria. 

A primary goal of the present set of studies was to determine whether LAMP tests would do 
as well or better than conventional paper-and-pencil tests in predicting meaningful learning criteria. Of 
course, the present studies were limited to learning in a single content area--that of electronic logic 
gates. Moreover, time permitted only a small sample of LAMP tests to be administered and validated. 
Nevertheless, results from the present research will add to an accumulation of evidence to be used to 
evaluate LAMP tests. 

In the logic gates task, two types of criterion data were considered. The first plausibly 
reflected individual differences in the acquisition of declarative knowledge and was indexed by the 
accuracy of specifying the outputs of logic gates. The second criterion reflected the acquisition of 
procedural skill, and was indexed by the time required for the subjects to correctly analyze and solve 
the logic gates problems. Each of these aspects of the criterion was treated separately in the following 
analyses. 

Acquisition of Declarative Knowledge 

The percent correct score for any one block of gate problems indicated the level of declarative 
knowledge for a subject at that stage of learning. Thus, it was desirable to relate ASVAB and LAMP 
tests to accuracy scores separately for each of the six blocks of gates without negations and for each of 
the four blocks with negations. There were two problems with this approach. First, as more and more 
individuals reach a level of perfect accuracy, the obtained validities suffered from a restriction in range 
due to ceiling effects. Second, individual block scores were based on a small number of problems, and 
were therefore somewhat unreliable. For this reason, the average latency across each series of blocks 
was also included as a criterion. 
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Two approaches were taken to test validation. Both involved computing differences in full- 

and restricted-model R 2 s. The initial step in the Approach 1 analysis was to include all ASVAB and 

LAMP tests in a single multiple regression equation to predict each criterion. The squares of these 
2 

multiple correlations (R s) reflected the amount of criterion variance accounted for by the combined 
test battery. Then the LAMP tests were removed from the equations to determine the losses in 
criterion variance accounted for. In each instance, the loss in R 2 indicated the unique contribution of 
the LAMP tests to the ASVAB tests in accounting for criterion variance. Next, the ASVAB tests were 

removed from the full equation to determine their unique contribution to LAMP tests in accounting for 

2 

criterion variance. In both instances, the significance levels of R losses were computed. The results 
from these analyses are reported in Table 6. 



Table 6. Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 1 Accuracy 
Criteria, Estimated by Approach 1 





ASVAB ADDS TO LAMP 


LAMP ADDS TO ASVAB 


FULL-MODEL 


BLK 


R 2 


F 


SIGN. 


R 


F 


SIGN. 


R 


R 2 


1-1 


.024 


1.397 


.179 


.131 


5.103 


.000 


.528 


.279 


1-2 


.017 


.997 


.445 


.153 


6.037 


.000 


.535 


.287 


1-3 


.010 


.600 


.814 


.180 


7.117 


.000 


.539 


.290 


1-4 


.016 


.904 


.530 


.168 


6.507 


.000 


.525 


.275 


1-5 


.022 


1.371 


.191 


.210 


8.667 


.000 


565 


.319 


1-6 


.006 


.345 


.968 


.186 


7.197 


.000 


.521 


.272 


Avl 


.017 


1.153 


.321 


.222 


10.319 


.000 


.628 


.394 


2-1 


.065 


4.269 


.000 


.133 


5.809 


.000 


.596 


.356 


2-2 


.036 


2.264 


.014 


.155 


6.421 


.000 


.567 


.321 


2-3 


.034 


2.184 


.018 


.174 


7.370 


.000 


.579 


.336 


2-4 


.025 


1.562 


.115 


.192 


8.071 


.000 


.574 


.329 


Av2 


.044 


3.008 


.001 


.182 


8.267 


.000 


.617 


.381 


AVG 


.035 


2.763 


.003 


.236 


12.412 


.000 


.682 


.465 



Notes: Block codes are defined in Figure 2. See text for explanation of Approach 1. 
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The combination of LAMP and ASVAB tests did a reasonably good job of predicting 
individual differences in learning efficiency, the obtained multiple correlations ranging from the low to 
the high Sffs for the various learning blocks. The multiple correlations for an average across blocks 
without negations was .628, for blocks with negations was .617, and for all blocks combined was .682 
(46.5 percent of the criterion variance being accounted for). Tests from the ASVAB did not add 
significantly to LAMP tests in predicting accuracy scores on lope gates without negations, and added 
very little to the LAMP tests in predicting learning on gates with negations. On the other hand, LAMP 
tests uniquely accounted for an additional 222% of the criterion variance for gates without negations, 
18.2% for gates with negations, and 23.6% for an average across all the problem blocks. 

The significance levels associated with losses in R s reported in Table 6 should be trustworthy, 

since appropriate degrees of freedom were used in the computations of F-ratios. However, the 

2 

estimated unique contributions, which were determined by subtracting restricted-model R s from full- 
model R 2 s, were likely to be somewhat inaccurate because of the varying number of independent 
variables in the equations. Other factors held constant, the larger the number of independent variables 
in an equation, the more likely the computed R will be an overestimate of the population parameter. 
Since the ASVAB equations contained only ten independent variables, while the LAMP equations 
contained fifteen, there was a greater opportunity for capitalization on chance relationships in the 
LAMP equations. There was even a greater opportunity for capitalization on chance relationships in 

the full-model, which contained 25 independent variables. To ameliorate these circumstances, 

2 

Approach 2 was taken, in which unique contributions were estimated using differences in R s which 
had been adjusted for the number of independent variables to more closely approximate the population 
parameters. 4 The results of the Approach 2 analyses are reported in Table 7. 

Table 7 reports the Rs and adjusted R 2 s for ASVAB tests alone, for LAMP tests alone, and 

for a combination of LAMP and ASVAB tests for the various gates-course accuracy criteria. The last 

column indicates the unique contribution of LAMP tests to the ASVAB in accounting for criterion 

2 

variance. 5 These values are based upon differences in the full- and restricted-model adjusted R s, and 
therefore should be closer to the population parameters. They indicate that the LAMP tests added 
20.5% unique variance to ASVAB in accounting for average accuracy scores across gate problems 



4 A sample R 2 usually does not fit the population as well as it fits the computing sample. The following 
equation was applied to adjust the R 2 to more closely reflect the goodness of fit of the model in the 
population: R 2 a =R 2 - (p(l-R 2 ))/(N-p-l), where p is the number of independent variables in the 
equation. 

5 In this, as well as in succeeding tables, LAMP/ASVAB and ASVAB/LAMP represent the 
contributions of LAMP to ASVAB and ASVAB to IjVMP, respectively. 
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without negations, 163% in accounting for average accuracy scores across gate problems with 
negations, and 222% in accounting for average accuracy scores across all ten problem blocks. 

While the LAMP tests contributed substantially to ASVAB tests in accounting for the 
individual differences in declarative learning as reflected in the accuracy scores, the data indicate that 
ASVAB tests added nothing to the LAMP tests in predicting performance on the test blocks without 
negations, and very little to the LAMP tests in predicting performance on the test blocks with 
negations. 



Table 7. Unique Contributions of ASVAB and LAMP tests to the Prediction of Study 1 Accuracy 
Criteria, Estimated by Approach 2 





FULL MODEL 


LAMP 


ASVAB 


LAMP/ 


ASVAB/ 


BLOCK 


R 


ADJR 


R 


ADJR 


R 


ADJR 


ASVAB 


LAMP 


1-1 


528 


.236 


.504 


.229 


385 


.129 


.107 


.007 


1-2 


535 


244 


.519 


.244 


.366 


.114 


.130 


.000 


1-3 


539 


248 


.529 


.255 


332 


.090 


.158 


-.007 


1-4 


525 


.232 


.509 


.234 


328 


.087 


.175 


-.002 


1-5 


.565 


.279 


.545 


.272 


.330 


.089 


.190 


.007 


1-6 


.521 


.229 


.516 


240 


.293 


.065 


.164 


-.011 


Avl 


.628 


.358 


.614 


356 


.415 


.153 


.205 


.002 


2-1 


596 


.317 


.539 


.266 


.412 


.205 


.112 


.051 


2-2 


.567 


.281 


.534 


.260 


.408 


.149 


.132 


.021 


2-3 


.579 


.296 


.549 


277 


.402 


.143 


.153 


.019 


2-4 


574 


.290 


.552 


.280 


370 


.117 


.173 


.010 


Av2 


.617 


.344 


.580 


.314 


.446 


.181 


.163 


.030 


AVG 


.681 


.433 


.656 


.410 


.478 


.211 


.222 


.023 



Notes. In those instances in which the adjusted R 2 for the LAMP tests is higher than that for the 
FULL MODEL, the estimated contribution for the ASVAB tests is negative. This is an artifact of the 
estimation procedure. Block codes are defined in Figure 2. See text for explanation of Approach 2. 

Acquisition of Procedural Skill 



In most instructional settings, the development of procedural skill tends to follow the 
acquisition of declarative knowledge. In the present study, individuals first learned the rules for 



ERIC 



19 



analyzing the input signals for logic gates in order to correctly specify their output signals. With 
practice in applying these rules, they were able to increase the speed of this analysis and decision 
activity. Thus, the development of procedural skill was reflected in the latency scores. Table 8 reports 
the results of an analysis to determine the unique contributions of LAMP and ASVAB tests in 
accounting for variance in these scores. It is similar to the Approach 1 data presented in Table 6 for 
accuracy scores. The multiple correlations are for the full model, using a combination of all ASVAB 
and LAMP tests, while the other columns present the unique contributions of ASVAB to LAMP tests 
and LAMP tests to ASVAB in accounting for the criterion variance. The F-ratios and significance 
levels are reasonable population estimates, since they were computed using appropriate degrees-of- 
freedom values. 



Table & Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 1 Latency 
Criteria, Estimated by Approach 1 





ASVAB ADDS TO LAMP 


LAMP ADDS TO ASVAB 


FULL-MODEL 


BLK 


R 2 


F 


SIGN. 


R 2 


F 


SIGN. 


R 


R z 


1-1 


.021 


1.13 


339 


.167 


5.847 


.000 


.443 


.197 


1-2 


.009 


.452 


.920 


.108 


3342 


.000 


.272 


.138 


1-3 


.011 


339 


.863 


.104 


3.413 


.000 


379 


.144 


1-4 


.008 


.430 


.932 


.154 


5.298 


.000 


.428 


.184 


1-5 


.014 


.723 


.703 


.147 


5.095 


.000 


.436 


.190 


1-6 


.014 


.763 


.665 


.186 


6.724 


.000 


.470 


.221 


Avl 


.009 


.493 


..895 


.167 


5.897 


.000 


.449 


.201 


2-1 


.005 


303 


.980 


.229 


8.482 


.000 


.490 


.240 


2-2 


.012 


.652 


.769 


.225 


8396 


.000 


.495 


.245 


2-3 


.009 


.462 


.914 


.199 


7.156 


.000 


.468 


.219 


2-4 


.016 


.887 


.545 


.207 


7.727 


.000 


.498 


.248 


Av2 


.010 


.597 


,817 


.247 


9.488 


.000 


.518 


.268 


AVG 


.006 


.324 


.975 


.220 


8.192 


.000 


.496 


.246 


Notes: See text for explanation of Approach 1. Block codes defined in Figure 2. 
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The data in Table 8 indicate that the ASVAB tests added nothing to the LAMP test in 
predicting the development of procedural skill. This was to be expected, since the ASVAB tests were 
designed to predict technical school grades, which are primarily a reflection of declarative knowledge 
acquisition. On the other hand, the LAMP tests added 16.7% unique variance to the ASVAB tests in 
predicting average latency scores on gates without negations, 24.7% in predicting the average on gates 
with negations, and 22.0% in predicting the average across all gate blocks. 

Again, these estimates are somewhat inaccurate because of the varying numbers of 
independent variables included the equations. Therefore, an Approach 2 analysis, based on adjusted 

2 

R s, is reported m Table 9. 



Table P. Unique Contributions of ASVAB and LAMP tests to the Prediction of Study 1 Latency 
Criteria, Estimated by Approach 2. 





FULL MODEL 


LAMP 


ASVAB 


LAMP/ 


ASVAB/ 


BLOCK 


R 


AD J R 


R 


AD J R 


R 


AD J R 


ASVAB 


LAMP 


1-1 


.443 


.149 


.418 


.146 


.172 


.007 


.142 


.015 


1-2 


.372 


.087 


.359 


.099 


.173 


.008 


.079 


-.012 


1-3 


379 


.093 


.364 


.103 


.199 


.018 


.075 


-.010 


1-4 


.428 


.135 


.419 


.147 


.172 


.008 


.127 


-.012 


1-5 


,436 


.142 


.420 


.147 


.208 


.021 


.121 


-.005 


1-6 


.470 


.174 


.454 


.179 


.185 


.012 


.162 


-.005 


Avl 


.449 


.154 


.438 


.164 


.184 


.012 


.142 


-.010 


2-1 


.490 


.195 


.455 


.208 


.105 


-.011 


.206 


-.013 


2-2 


.495 


.200 


.483 


.206 


.139 


-.003 


.203 


-.006 


2-3 


.468 


.173 


.459 


.183 


.144 


-.002 


.175 


-.010 


2-4 


.498 


.203 


.482 


.205 


.203 


.019 


.184 


.002 


Av2 


.518 


.225 


.508 


.232 


.147 


-.007 


.232 


-.007 


AVG 


.4% 


.201 


.490 


.213 


.169 


.004 


.197 


-.012 



Notes. In those instances in which the adjusted R 2 for the LAMP tests is higher than that for the 
FULL MODEL, the estimated contribution for the ASVAB tests is negative. Block codes are defined 
in Figure 2. See text for explanation of Approach 2. 
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The Approach 2 data in Table 9 tell essentially the same story. The ASVAB tests did not 
contribute to the LAMP tests in accounting for the latency criteria. As a matter of fact, in most 
instances the estimated unique contribution of ASVAB tests was a negative value (due to imperfections 
in out estimation procedures) which occurred when the adjusted R 2 for the LAMP tests was higher 
than that for the FULL MODEL. On the other hand, the LAMP tests added substantially to ASVAB 
tests in predicting these criteria. The LAMP contributions were estimated to be 14.2% for an average 
across gates without negations, 23.2% for an average across gates with negations, and 19.7% across all 
gate blocks. 

As mentioned previously, procedural skill in this study was reflected in the time required for 
subjects to analyze the input signals of logic gates and correctly determine their output signals. 
However, in Block 1 some individuals made a number of errors in specifying output signals, indicating 
that their understanding of the rules was weak at that stage. The meaning of latency scores is unclear 
when accuracy is low, and such scores are likely to be unpredictable by ability factors. In a sense, they 
introduced a type of 'error variance' in the criterion scores. This problem becomes less and less 
pronounced in the later blocks, in which most responses were correct. 

One approach to clarifying latency data is to discard subjects who do not achieve some 
minimum level of declarative knowledge during the course. This procedure tends to purify the latency 
scores as a skill acquisition criterion. In the present study, subjects were eliminated who did not 
achieve at least a 90% accuracy level in the fourth block of gates without negations. Thus, 104 subjects, 
were eliminated, leaving 344 subjects in the residual sample. The average accuracy levels for these 344 
subjects across the six blocks of gates without negations was 81.8%, 90.6%, 94.2%, 96.4%, 96.6%, and 
96.9%, respectively, and 86.5%, 90.0%, 92.3%, and 925% for the gates with negations, respectively. 
Tables 10 and 11 report data comparable to that reported in Tables 8 and 9, except that they are based 
on subjects in the residual sample. 

The multiple correlations in Table 10 are higher than they were in Table 8, since there was 
less variance associated with incorrect responses in the residual sample data. However, the conclusions 
based upon these results are the same. The ASVAB tests added nothing to LAMP tests in accounting 
for procedural skill development as reflected in the latency data. On the other hand, LAMP tests 
added considerable variance to the ASVAB tests in accounting for procedural skill development. The 
unique contribution of LAMP tests was estimated to be 22.8% for an average across the blocks of gates 
without negations, 24.2% for an average across blocks with negations, and 24.0% for an average across 
all blocks. 
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The data in Table 11 represent similar findings, except that the estimated LAMP contribution 
levels are slightly lower, being 16*5% for the blocks without negations, 21.7% for the blocks with 
negations, and 213% across all blocks. 



Table 10. Unique Contributions of AS VAB and LAMP Tests to the Prediction of Study 1 Latency 
Criteria, Estimated by Approach 1 in the Residual Sample (N=344) 



ASVAB ADDS TO LAMP LAMP ADDS TO ASVAB FULL-MODEL 
BLK R 2 F SIGN. R 2 F SIGN. R R 2 



1-1 


.044 


1.815 


.057 


.193 


5356 


.000 


.486 


.236 


1-2 


.032 


1324 


.216 


.138 


3.756 


.000 


.469 


.220 


1-3 


.018 


.744 


.682 


.163 


4.555 


.000 


.489 


.239 


1-4 


.010 


.409 


.942 


.210 


6.005 


.000 


.508 


.258 


1-5 


.014 


.636 


.783 


.212 


6.293 


.000 


.534 


.285 


1-6 


.014 


.630 


.788 


.243 


7.298 


.000 


.542 


.239 


Avl 


.018 


.842 


.589 


.228 


6.939 


.000 


.550 


.303 


2-1 


.019 


.811 


.618 


.216 


6.286 


.000 


.522 


.272 


2-2 


.010 


.447 


.923 


.217 


6.402 


.000 


.531 


.282 


2-3 


.011 


.467 


.911 


.193 


5.475 


.000 


.504 


.254 


2-4 


.020 


.927 


.509 


.231 


7.009 


.000 


.550 


.302 


Av2 


.017 


.798 


.630 


.242 


7.500 


.000 


.561 


.315 


AVG 


.019 


.879 


.553 


.240 


7.375 


.000 


.558 


.311 



Notes: See text for explanation of Approach 1. Block codes are defined in Figure 2. 
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Table 11 Unique Contributions of ASVAB and LAMP tests to the Prediction of Study 1 Latency 
Criteria, Estimated by Approach 2 in the Residual Sample (N = 344) 





FULL MODEL 


LAMP 




ASVAB 


LAMP/ 


ASVAB/ 


BLOCK 


R 


AD J R 2 


R 


ADJR 2 


R 


ADJR 


ASVAB 


LAMP 


1-1 


.486 


.176 


.439 


.156 


208 


.015 


.161 


.020 


1-2 


.469 


.159 


.433 


.150 


.286 


.082 


.077 


.009 


1-3 


.489 


.179 


.470 


.186 


.275 


.048 


.131 


-.007 


1-4 


.508 


.200 


.498 


.214 


.219 


.019 


.181 


-.014 


1-5 


.534 


.229 


501 


.238 


.271 


.045 


.184 


-.014 


1-6 


.542 


.239 


.529 


.247 


.226 


.023 


.216 


-.008 


Avl 


.550 


.212 


.533 


.252 


.273 


.047 


.165 


-.040 


2-1 


.522 


.215 


503 


.220 


.238 


.028 


.187 


-.005 


2-2 


.531 


.226 


521 


.239 


.255 


.037 


.188 


-.013 


2-3 


.504 


.196 


.493 


.209 


.248 


.033 


.163 


-.013 


2-4 


.550 


.247 


531 


.249 


.267 


.043 


..204 


-.002 


Av2 


.561 


.261 


.546 


.266 


.270 


.044 


.217 


-.005 


AVG 


.558 


.257 


540 


.260 


.268 


.044 


.213 


.018 



Notes. In those instances in which the adjusted R 2 for the LAMP tests is higher than that for the 
FULL MODEL, the estimated contribution for the ASVAB tests is negative. Block codes are defined 
in Figure 2. See text for explanation of Approach 2. 
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STUDY 2 



Study 2 involved several modifications of the gates course and in the LAMP tests selected for 
validation. To provide a longer practice period for the subjects to improve their procedural skills, the 
course was modified to include eight blocks of gates without negations instead of only six In addition, 
complex gate circuit problems were included to increase job relevance. The inclusion of more learning 
blocks reduced the time available for predictor test administration, so the Number Fact Retrieval, 
Meaning Identity, 123 Number Reduction, and Dipt Span tests were dropped in order to increase the 
number of non-quantitative tests defining the working memory factor; to include measures of spatial 
abilities and associative learning abilities, four new tests were added: General Knowledge, Word 
Association, Figure Recognition, and Mental Character Generation. Finally, a pretest of 'going-in' 
knowledge was administered to subjects. This pretest was made up from problems selected from the 
course to follow. To decrease the probability of subjects recording the correct answers by chance, six 
incorrect but plausible question alternatives (foils) were generated and included as possible answers to 
each question. Scores from this pretest were at chance level (M=14.6, S.D.=12.1), and they did not 
correlate significantly with any of the accuracy or latency blocks in Study 2. It was concluded that the 
course represented a relatively novel learning situation to the subjects. 

Table 12 reports the mean percent correct and the mean latency scores for the eight blocks of 
simple gates without negations and for averages across blocks in other sections of the course. The 
development of procedural skill is clearly reflected in the systematic reduction of response times across 
the eight blocks of simple gates without negations; the mean of median latencies dropped from 2.21 
seconds in Block 1 to 1.23 seconds in Block 8, with a corresponding drop in standard deviations from 
.69 seconds in Block 1 to .32 seconds in Block 8. 

The percent correct scores also reflect a systematic strengthening of declarative knowledge 
across these same blocks, with the means increasing from 83.7 in Block 1 to 97.4 in Block 8. There was 
an increasing restriction in range due to ceiling effects, which was more serious than that encountered 
in Study 1. By the 8th block, 50% of the subjects made perfect scores, and another 29% missed only 
one problem. The standard deviation of the scores in Block 8 was only 4.0%. The effect of the ceiling 
effect was to render the later blocks less valuable as measures of individual differences in knowledge 
acquisition. At the same time, since most subjects made perfect or near perfect scores in the later 
blocks, the latency scores in these blocks were wellsuited as indicators of their level of procedural skill. 
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Table 12. Means and Standard Deviations for Study 2 Blocks And Averages 



ACCURACY LATENCY 
MEANS SX>.'s MEANS S.D's 



1-1 


83.67 


11.62 


2.21 


.69 


1-2 


89.15 


11.23 


1.83 


55 


1-3 


92.66 


9.40 


1.63 


.56 


1.4 


95.02 


7.19 


1.46 


.48 


1-5 


95.86 


5.80 


134 


.42 


1-6 


96.12 


5.75 


132 


38 


1-7 


96.83 


4.73 


1.25 


.JO 


1 ft 


07 

y i .jj 


402 


1.23 


.32 


Avl 


9333 


6.06 


1.46 


38 


2-1 


87.25 


14.91 


333 


.80 


Av3 


91.47 


9.94 


230 


.61 


Av4 


89.13 


11.79 


3.47 


1.06 


AVG 


91.51 


7.89 


2.01 


.46 



Notes. Block codes are defined in Figure 2. 



Factor Analysis 

In a factor analysis of the predictor tests, six principal axis factors were extracted and rotated 
using the Varimax method. The results of this analysis are reported in Appendix B. 

The six factors were given the same names as the six factors identified in Study 1. They were 
essentially equivalent, except for the Working Memory factor, which was now defined by verbal and 
spatial tests as well as by quantitative tests, and for the Processing Speed factor, which was a little 
weaker because the Meaning Identity test had been dropped from the battery. Also, the Numerical 
Facility/Perceptual Speed factor contained less numerical variance because of the omission of the 
Number Fact Retrieval test. The correlations between factor scores and the various gates course 
accuracy criteria are presented in Table 13, while those for the course latency criteria are presented in 
Table 14. 
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Table 13. Correlations of Factor Scores with Study 2 Accuracy Criteria 


DLIi 


VFPRAI 


VY IViVlCiVl 




PR OCA CC 


TECHKNOW 


NIIM PS 


1 1 
1-1 


.JO 




no 


.ZZ 






1 1 

1-Z 


1 1 
• 1Z 




no 

-.uo 


u3 


07 
.u / 


1 1 
.11 


1-3 


.04 


.50 


-.07 


.24 


.01 


.10 


1-4 


.06 


.45 


-.08 


.24 


.04 


.08 


1-5 


.01 


.37 


-.09 


.23 


.10 


.11 


1-0 


.03 


31 


-.01 


.21 


-.01 


.oy 


1-7 


-.01 


.28 


-.06 


.18 


-.04 


.09 


1-8 


-.02 


.26 


-.09 


.21 


-.08 


.12 


Avl 


.08 


.53 


-.09 


.28 


.03 


.13 


2-1 


.14 


.43 


-.05 


.22 


.17 


.12 


Av3 


.15 


.49 


-.11 


.28 


.12 


.05 


AVG 


.15 


.54 


-.11 


.30 


.12 


.11 



Notes. Block codes are defined in Figure 2. 



The criteria in these, as well as in later tables, include eight blocks of simple gates without 
negations, an average across these eight blocks, a single block of simple gates with negations, an 
average for the two blocks of mixed simple gates (with and without negations), an average across the 
three blocks of mixed simple gates and complex gate circuits (with and without negations), and, finally, 
an average across all of the course blocks. 

Data in Table 14 reveal that the Working Memory factor scores had highest relationships with 
all of the accuracy criterion variables, which is consistent with the findings in Study 1. It should be 
noted, however, that the magnitude of the relationships dropped systematically from Block 1 to Block 
8 of simple gates without negations. This downturn was expected, and is attributed to the decreasing 
individual differences variance due to ceiling effects. The correlation of the Working Memory factor 
scores with an average across all blocks of simple gates without negations was .530, and was .543 for an 
average across all blocks in the course. 




Table 14. Correlations of Factor Scores with Study 2 Latency Criteria 



BLK VERBAL WKMEM PROCSPD PROCACC TECHKNOW NUM PS 



1-1- 


-.09 


.09 


.19 


.10 


-.06 


no 
-.uy 


1-2 


-.17 


-.14 


.28 


.02 


-.12 


-.14 


1 1 




.10 


V\ 


-.03 


-.06 


-.12 


1-4 


-.11 


-.28 


21 


-.05 


-.05 


-.10 


1-5 


-.12 


-34 


31 


-.07 


-.09 


-.07 


1-6 


-.12 


-30 


.40 


-.06 


-.07 


-.10 


1-7 


-.14 


-32 


.35 


-.04 


-.06 


-.14 


1-8 


-.16 


-.30 


37 


-.02 


-.05 


-.12 


Avl 


-.15 


-.29 


39 


-.02 


-.07 


-.13 


2-1 


-.09 


-.15 


.26 


.10 


-.11 


-.13 


Av3 


-.14 


-.11 


35 


.11 


-.05 


-.15 


Av4 


-.08 


.09 


.21 


.14 


-.02 


-.21 


AVG 


-.16 


-.13 


.36 


.09 


-.06 


-.16 



Notes. Block codes are defined in Figure 2. 



Again consistent with the findings in Study 1, the Processing Accuracy factor scores had the 
second highest relationships with accuracy scores from the gates course. The Numerical 
Facility/Perceptual Speed, Verbal and Technical Knowledge factor scores had low but significant 
relationships with scattered test blocks, correlating -.110, .145, and .117, respectively, with the average 
accuracy across all course blocks. The data in Table 14 show that the Processing Speed factor scores 
had the highest relationships with the latency criteria, again consistent with the findings in Study 1. The 
Working Memory factor scores were particularly strong in predicting latencies in the later blocks of 
simple gates without negations, thus replicating another finding in Study 1. This finding is consistent 
with the hypothesis that individuals with high working-memory capacity attained sufficient strength in 
declarative knowledge in the early blocks to facilitate their development of procedural skill. The 
Numerical Facility /Perceptual Speed and the Verbal factor scores also had scattered relationships with 
latency criteria, attaining correlations of -.160 and -.155 respectively with the average latency across all 
blocks. 

In summary, the results of the factor analysis in Study 2 were remarkably similar to those 
obtained in Study 1, in spite of changes in the criterion and predictor variables. The same number of 
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factors was obtained in both studies, and the patterns of factor-criterion relationships were highly 
similar. 

Comparative Validities ofASVAB and LAMP Tests for Gates-Course Learning Criteria 
Acquisition of Declarative Knowledge 

As was the case in Study 1, the accuracy scores from the gates-course test blocks were 
accepted as indicators of acquired declarative knowledge. The same two approaches were taken to 
determine the ability of LAMP tests alone, ASVAB tests alone, and a combination of LAMP and 
ASVAB tests to predict these scores. In Approach 1, the losses in R 2 s were determined as LAMP tests 
and ASVAB tests were removed one set-at-a-time from full models containing both classes of 
predictors. In each instance, the unique contribution of the two predictor sets were computed, along 
with F-ratios and significance levels based on appropriate degrees of freedom. The results from this 
first analysis are reported in Table IS. 



Table 15. Unique Contributions of ASVAB and LAMP Tests to the Prediction of Study 2 Accuracy 
Criteria, Estimated by Approach 1 





ASVAB ADDS TO LAMP 


LAMP ADDS TO ASVAB 


FULL-MODEL 


BLK 


R 


F 


SIGN. 


R 


F 


SIGN. 


R 


R 


1-1 


.043 


2.764 


.003 


.155 


7.166 


.000 


.611 


.373 


1-2 


.039 


2.376 


.010 


.146 


6.428 


.000 


.585 


.342 


1-3 


.051 


3.047 


.001 


.158 


6.683 


.000 


.561 


.315 


1-4 


.042 


2394 


.009 


.152 


6.214 


.000 


.541 


.293 


1-5 


.022 


1.163 


.314 


.127 


4.742 


.000 


.474 


.224 


1-6 


.025 


1.243 


.261 


.105 


3.699 


.000 


.422 


.178 


1-7 


.028 


1.372 


.191 


.091 


3.163 


.000 


.404 


.163 


1-8 


.039 


1.921 


.041 


.094 


3.315 


.000 


.418 


.175 


Avl 


.042 


2.770 


.003 


.189 


8.929 


.000 


.622 


.387 


2-1 


.053 


3.120 


.000 


.120 


5.017 


.000 


.553 


.305 


Av3 


.055 


3.310 


.000 


.136 


5.901 


.000 


.575 


.330 


Av4 


.016 


1.035 


.413 


.240 


11.432 


.000 


.625 


.391 


AVG 


.031 


2.215 


.016 


.222 


11393 


.000 


.659 


.434 



Notes: Block codes defined in Figure 2. See text for explanation of Approach L 
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In Approach 2, multiple correlations were computed for the LAMP tests alone, the ASVAB 

tests alone, and for a combination of LAMP and ASVAB tests against the various accuracy criteria. 

2 

The unique contribution of each battery to the other was estimated by the differences in R s computed 
in the restricted and full models, after such R s had been adjusted for the number of independent 
variables to be more accurate estimates of population parameters. The results of this second analysis 
are reported in Table 16. 



Table 16. Unique Contributions of ASVAB and LAMP tests to the Prediciton of Study 2 Accuracy 
Criteria, Estimated by Approach 2 i 





FULL MODEL 


LAMP 


ASVAB 


LAMP/ 


ASVAB/ 


BLOCK 


p 


A TIT R 2 


R 


ADJ R 2 


R 


ADJ R 


ASVAB 


LAMP 


1-1 


.611 


336 


.575 


308 


.476 


.200 


.136 


.028 


1-2 


.585 


.303 


.551 


.280 


.443 


.177 


.126 


.023 


1-3 


561 


.274 


.513 


.238 


.396 


.136 


.138 


.036 


1-4 


541 


.251 


501 


.226 


.376 


.121 


.130 


.025 


1-5 


.474 


.179 


.450 


.175 


.312 


.076 


.103 


.004 


1-6 


.422 


.130 


391 


.124 


.271 


.051 


.079 


.006 


1-7 


.404 


.114 


.376 


.106 


.258 


.050 


.064 


.008 


1-8 


.418 


.126 


369 


.107 


.284 


.059 


.089 


.019 


AvI 


.622 


.351 


588 


324 


.446 


.180 


.171 


.027 


2-1 


553 


.264 


.502 


.227 


.430 


.166 


.098 


.037 


Av3 


515 


.291 


525 


131 


.441 


.175 


.116 


.040 


Av4 


.625 


.355 


.612 


354 


.388 


.130 


.225 


.001 


AVG 


.659 


.400 


.635 


.383 


.460 


.193 


.207 


.017 



Notes. Block codes are defined in Figure 2. See text for explanation of Approach 2. 

The impact of the ceiling effects on the eight blocks of simple gates without negations is 
reflected in Tables 15 and 16 by a systematic reduction in the magnitudes of the full-model Rs from 
.611 in Block 1 to .418 in Block 8. The unique contribution of ASVAB to LAMP tests in the prediction 
of these criteria was small, and in some instances not significant. On the other hand, the LAMP tests 
contributed significantly to the ASVAB tests in predicting all blocks in the course, including the blocks 
of simple gates. In spite of the ceiling effects, the estimated contribution of LAMP to ASVAB tests in 
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predicting average accuracy across the first eight blocks was estimated to be 18.9% by Approach 1 
(Table 15) and 17.1% by Approach 2 (Table 16). The LAMP tests were particularly powerful in 
predicting average accuracy scores across the blocks which contained complex gate circuits, yielding a 
multiple correlation of .612 compared with 388 for the ASVAB tests. In this instance, the unique 
contribution of the LAMP tests to ASVAB was estimated to be 24.0% by Approach 1, and 22,5% by 
Approach 2. The unique contribution of ASVAB tests to LAMP tests was estimated to be non- 
significant (F-ratio = 1.035, P> .4). The multiple correlation of the LAMP tests with average accuracy 
across the entire course was .635, compared with .460 for the ASVAB tests, and the unique 
contribution of LAMP tests to ASVAB was estimated to be 22.2% by Approach 1 and 20.7% by 
Approach 2. These estimates are to be compared with the 3.1% and 1.7% estimates of the 
contribution of ASVAB tests to LAMP tests, which were not significant at the .01 level. All of these 
values are remarkably similar to those obtained in Study 1. 



Acquisition of Procedural Skill 

As in Study 1, latency scores were used as indicants of procedural skill. Once individuals had 
the rules for solving gate problems firmly in mind, they tended to accurately apply them with increasing 
speed and efficiency. For most individuals, proceduralization of rules began even before they reached 
the Block 1 problem set. Some individuals made perfect scores on all problems in this block, and 
solved each problem with increased speed. Others were relatively accurate on AND or OR gates, but 
were still somewhat confused and inaccurate in solving NOR gates. The increase in accuracy from 
Block 1 to Block 8 reflected the continued acquisition and strengthening of declarative knowledge 
(concerning gate rules) within the subject sample, and by the 8th block, a 97% accuracy level was 
obtained. Latency scores were the best- measures of individual differences in procedural skill at any 
learning stage, but this criterion was contaminated to the extent that declarative learning had not been 
achieved. For this reason, the latency scores in Study 2 were analyzed in the same manner as they were 
in Study 1. First, they were analyzed in the full sample, using the same two approaches as were used 
for analyzing accuracy scores. The results of these analyses are reported in Tables 17 and 18. Then 
they were analyzed on a subsample of individuals who reached the 90% accuracy level by the 4th block 
of simple gates without negations. The results of these analyses are presented in Tables 19 and 20. 



39 

31 



Table 17. Unique Contributions of ASVAB and.LAMP Tests to the Prediction of Study 2 Latency 





ASVAB ADDS TO LAMP 


LAMP ADDS TO ASVAB 


FULL-MODEL 


BLK 


R 2 


F 


SIGN. 


R 


F 


SIGN. 


R 


R Z 


1-1 




1 9J\1 

JL.OO / 


048 


097 


3.272 


.000 


380 


.145 


1-2 


.024 


1.202 


288 


.131 


4.754 


.000 


.450 


.203 


1-3 


.014 


.744 


.673 


.152 


5.821 


.000 


.490 


.240 


1-4 


.024 


1.231 


.269 


.135 


4.950 


.000 


.457 


.209 


1-5 


.028 


1.607 


.102 


.194 


7.842 


.000 


.530 


.281 


1-6 


.022 


1.251 


.256 


.213 


8.823 


.000 


.547 


.301 


1-7 


.012 


.668 


.754 


.172 


6.827 


.000 


.517 


.267 


1-8 


.011 


.620 


.796 


.182 


7.261 


.000 


.522 


.272 


Avl 


.019 


1.001 


.361 


.205 


8.421 


.000 


.542 


.294 


2-1 


.025 


1.249 


.258 


.143 


5.188 


.000 


.448 


201 


Av3 


.022 


1.169 


.310 


.171 


6.486 


.000 


.487 


.237 


Av4 


.017 


.945 


.491 


.214 


8.334 


.000 


.505 


.256 


AVG 


.020 


1.113 


.350 


.206 


8.260 


.000 


.526 


.277 



Notes: Block codes defined in Figure 2. See text for explanation of Approach 1. 



The results reported in Tables 17 and 18 are comparable to those obtained in Study 1. In no 
instance did the ASVAB tests add unique valid variance (at the .01 level) to the LAMP tests in 
predicting latency scores. On the other hand, the unique contribution of LAMP tests to ASVAB in 
predicting the average latency on simple gates without negations was estimated by the two approaches 
to be 20.5% and 18.4%, respectively. The multiple correlation of LAMP tests for the average latency 
scores across all gate blocks was .507, compared with .267 for ASVAB tests, and the Approach 1- 
Approach 2 estimates of the unique contribution of LAMP tests to ASVAB tests in predicting this 
criterion were 20.6% and 18.5%, respectively. 
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Table 18. Unique Contributions of AS VAB and LAMP tests to the Prediction oJ Study 2 Latency 
Criteria, Estimated by Approach 2 





FULL MODEL 


LAMP 


ASVAB 


LAMP/ 


ASVAB/ 


BLOCK 


R 


ADJR 


R 


ADJR 


R 


ADJR 


ASVAB 


LAMP 


1-1 


380 


.094 


324 


.075 


.219 


.025 


.069 


.019 


1-2 


.450 


.156 


.423 


.151 


.268 


.050 


.106 


.005 


1-3 


/ion 




.*»/o 


.ZUU 


.Zy 1 


.UOO 


.130 


-.004 


1-4 


.457 


.162 


.430 


.157 


.272 


.052 


.110 


.005 


1-5 


.530 


.239 


.503 


.228 


.295 


.065 


.174 


.011 


1-6 


J549 


260 


.529 


.255 


297 


.067 


.193 


.005 


1-7 


.517 


.224 


.505 


.230 


.308 


.073 


.151 


-.006 


1-8 


522 


.229 


511 


.236 


300 


.069 


.160 


-.007 


Avl 


542 


.252 


.524 


.251 


.298 


.068 


.184 


.001 


2-1 


.448 


.154 


.420 


.149 


.241 


.036 


.118 


.005 


Av3 


.487 


.192 


.464 


.189 


.259 


.045 


.147 


.003 


Av4 


505 


.212 


.488 


.2D 


.205 


.019 


.167 


-.001 


AVG 


526 


.234 


.507 


.232 


.267 


.049 


.185 


.002 



Notes. In those instances in which the adjusted R 2 for the LAMP tests is higher than that for the 
FULL MODEL, the estimated contribution for the ASVAB tests is negative. Block codes are defined 
in Figure 2. See text for explanation of Approach 2. 

Discarding subjects who did not reach the 90% accuracy level by the 4th block of simple 0 ates 
without negations resulted in a loss of only 63 subjects, or 14.6% of the original sample. However, this 
smaller sample resulted in criterion measures which were considerably less contaminated by inaccurate 
responses. By the 4th block of simple gates without negations, the average percent correct score was 
over 97%, and this average rose to above the 98% level by the 8th block. Furthermore, in none of 
blocks of the more complex gates, which followed, did the average accuracy level fall below the 90% 
level. 

As expected, the full-model multiple correlations were higher in the residual sample, rising 
from .404 in Block 1 of simple gates without negations to 549 in Block 8. The full-model multiple 
correlation was .569 for an average across the eight blocks without negations, and .572 for an average 
across all course blocks. Again, the data in Tables 19 and 20 indicate that the ASVAB tests did not add 
significantly to the LAMP tests in predicting any of the latency criteria. On the other hand, using the 
two approaches previous described, the estimated contribution of LAMP tests to ASVAB tests was 
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21.8% and 19.6% for predicting the average latency across simple gates without negations, and 21.9% 
and 19.7% for the average latency across all course blocks. 



Table 19. Unique Contributions of AS VAB and LAMP Tests to the Prediction of Study 2 Latency 
Criteria, Estimated by Approach 1 in the Residual Sample (N=368) 





ASVAB ADDS TO LAMP 


LAMP ADDS TO ASVAB 


FULL-MODEL 


BLK 


K 


r 




R 2 


F 


SIGN. 


R 


R 


1-1 


.036 


1.505 


.136 


.111 


3.257 


000 


.404 


.163 


1-2 


.028 


1.312 


.222 


.148 


4.908 


.000 


.511 


.261 


1-3 


.027 


1.282 


.239 


.159 


5.384 


.000 


.524 


.274 


1-4 


.031 


1.425 


.167 


.162 


5360 


.000 


.508 


.258 


1-5 


.026 


1.291 


.234 


204 


7.111 


.000 


.547 


.299 


1-6 


.027 


1341 


.207 


.206 


7.266 


.000 


.553 


.306 


1-7 


.011 


.499 


.890 


.182 


6.137 


.000 


.525 


.275 


1-8 


.012 


.567 


.841 


.194 


6.803 


.000 


.549 


.302 


Avl 


.023 


1.149 


.325 


.218 


7.893 


.000 


.569 


.324 


2-1 


.019 


.910 


.524 


.198 


6.774 


.000 


.530 


.281 


Av3 


.017 


.874 


558 


.211 


7.735 


.000 


.577 


.333 


Av4 


.019 


.892 


.541 


.188 


6.433 


.000 


.531 


.282 


AVG 


.018 


.908 


.525 


.219 


7.953 


.000 


.572 


.327 



Notes: Blo-Jc codes are defined in Figure 2. See text for explanation of Approach 1. 
Discussion 

All in all, the results from Study 2 were very similar to those obtained in Study 1. The ASVAB 
tests were found to be poor predictors of latency scores in both studies, and they added nothing to the 
LAMP tests in predicting such criteria. This result was not unexpected, since the ASVAB was designed 
to predict technical school grades and has historically been validated against grade criteria. On the 
other hand the LAMP tests produced a multiple correlation of .490 in Study 1 and .507 in Study 2 in 
predicting the average latency scores in the full samples. When the samples were residualized to 
include only those subjects who had accuracy scores of at least 90% on Block 4 of simple gates without 
negations, the multiple correlations were computed to be .558 in Study 1 and .572 in study 2. 
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Table 20. Unique Contributions of ASVAB and LAMP tests to the Prediction of Study 2 Latency 
Criteria, Estimated by Approach 2 in the Residual Sample (N = 368) 





FULL MODEL 


LAMP 


ASVAB 


LAMP/ 


ASVAB/ 


BLOCK 


R 


ADJR 


R 


ADJR" 


R 


ADJR 


ASVAB 


LAMP 


1-1 


.404 


.105 


356 


.092 


.229 


.026 


.079 


.013 


1-2 


.511 


.209 


.482 


.202 


336 


.088 


.077 


.007 


1-3 


324 


.224 


.497 


.217 


.339 


.090 


.134 


.007 


1-4 


.508 


.206 


.476 


.196 


309 


.070 


.136 


.010 


i-j 


541 


.250 


572 


.244 


309 


.070 


179 


.006 


1-6 


553 


.258 


528 


.250 


.317 


.075 


.183 


.008 


1-7 


525 


.224 


.514 


.235 


306 


.068 


.156 


-.011 


1-8 


549 


.253 


539 


.262 


.329 


.083 


.170 


-.009 


Avl 


569 


.277 


549 


.274 


.326 


.081 


.196 


.003 


2-1 


.530 


.231 


512 


.232 


.288 


.057 


.174 


-.001 


Av3 


.577 


.286 


.562 


.289 


.350 


.098 


.188 


-.063 


Av4 


531 


.232 


513 


.234 


306 


.068 


.164 


-.004 


AVG 


511 


.280 


.556 


.282 


.329 


.083 


.197 


-.002 



Notes. In those instances in which the R 2 for the LAMP tests is higher than that for the FULL 
MODEL, the estimated contribution for the ASVAB tests is negative. Block codes are defined in 
Figure 2. See text for explanation of Approach 2. 

The multiple correlations of LAMP tests for the average accuracy scores across all blocks was 
.681 in Study 1 and .659 in Study 2. The comparable values obtained with ASVAB tests were .478 and 
.460, respectively. To put it another way, the LAMP tests accounted for more than twice the amount of 
criterion variance as was accounted for by the ASVAB tests. The unique contribution of LAMP tests 
to ASVAB tests in predicting the overall accuracy criteria was estimated to be at least 20% in both 
studies, using either of the described estimation approaches. 



43 



Decision on Need for Study 3 



There are two factors which should be considered before accepting the findings in Studies 1 
and 2. One deals with common-methods, and the other with concurrence. In the present studies, the 
LAMP tests and the gates course criterion tests were both presented by microcomputers, while the 
ASVAB tests were administered using paper-and-pencil procedures. The common methods may have 
given an advantage to the LAMP tests. This matter will be considered in the General Discussion 
section. The other factor concerns the fact that the LAMP tests and the course criterion tests were 
administered concurrently (in the same session), while the ASVAB tests had been administered some 
weeks or months previously in the operational testing stations. It is reasonable to expect that abilities 
measured at the time of course administration would be more valid predictors of course outcome than 
would be abilities measured at some prior point in time. This matter has been discussed and evaluated 
in at least two prior studies. Christal (1989) conducted a study based on several thousand subjects and 
found a small but significant concurrency effect. However, this study used ASVAB tests both as 
predictors and criteria. Divigi (1990) conducted a study to evaluate the incremental validity added by 
experimental tests to ASVAB in predicting job knowledge and hands-on-performance criteria in four 
occupations. In this study he used original ASVAB scores and scores from a version of the ASVAB 
administered concurrently with the experimental tests. He found the difference between incremental 
variance using concurrent and enlistment ASVABs to be "minor compared to variations across 
different criterion variables and occupational specialities." 

In spite of the evidence that concurrency effects were not likely to be large, a decision Was 
made to conduct a third study to specifically test for them in the context of validating ASVAB tests 
against criteria from the logic gates courses. 



STUDY 3 

With only three hours of testing time available, it was not possible to readminister the entire 
ASVAB along with a gates course. However, results from the factor analyses in both Study 1 and in 
Study 2 indicated that most of the predictive validity for the accuracy criteria came from measures of 
working memory. Since Arithmetic Reasoning, Mathematics Knowledge, and Mechanical 
Comprehension were the only ASVAB tests with significant loadings on this factor, analyses were 
conducted using Study 1 and Study 2 data to compare the multiple correlations of these three tests >vith 
those from the entire ASVAB in predicting the various accuracy criteria. The results from these 
analyses are presented in Table 21. 
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Table 21. Comparative Multiple Correlations of. the Entire ASVAB and a Subset of ASVAB Tests for 
Accuracy Criteria in Study 1 and Study 2 



ERIC 



STUDY 1 STUDY 2 

ASVAB SUBSET* ASVAB SUBSET 



* 



1-1 381 .381 .460 .452 

1-2 359 359 .424 .424 

1-3 315 315 376 359 

1-4 .295 .295 363 363 

1-5 .310 .297 .273 .273 

1-6 .255 .255 .238 .238 



Avl .255 .255 

1-7 .215 .215 

1- 8 .264 .243 

Avl .431 .431 

2- 1 .447 .429 .415 .415 
2-2 .388 378 

2-3 391 .381 

2-4 352 352 

Av2 .433 .421 

Avg3 .428 .416 

Avg4 .360 .360 

AVG .464 .460 .455 .455 



Notes. * Subset of ASVAB tests is composed of Arithmetic Reasoning, Mathematics Knowledge, and 
Mechanical Comprehension. Block codes are defined in Figure 2. 
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The data in Table 21 confirm that essentially all of the criterion variance predicted by ASVAB 
could be captured by the three tests mentioned; therefore they were administered along with the Study 
2 gates course to 312 airmen in the 11th day of basic military training. In order to gauge the magnitude 
of concurrency ejects, multiple correlations for each of the course accuracy criteria were computed 
twice; once using scores on the three tests obtained from the operational testing files, and again, using 
scores on the three tests from the concurrent (retest) administration. The results of these analyses are 
presented in Table 22. 

Table 22. Predictive and Concurrent Multiple Correlations of a Subset of ASVAB tests for Study 2 
Accuracy Criteria 



BLK PREDICTIVE CONCURRENT 



1-1 


.456 


.456 


1-2 


.478 


.458 


1-3 


.455 


.453 


1-4 


362 


334 


1-5 


338 


.309 


1-6 


.290 


.284 


1-7 


.251 


.249 


1-8 


.206 


.249 


Avl 


.453 


.433 


2-1 


.523 


.514 


Av3 


.498 


.431 


Av4 


.495 


.477 


AVG 


.534 


.505 



Notes. *The subset of ASVAB tests was composed of Arithmetic Reasoning, Mathematics Knowledge, 
and Mechanical Comprehension. Block codes are defined in Figure 2. 



The data in Table 22 reveal little or no differences in the multiple correlations based on the 
original and concurrent test scores. Where differences did exist, they tended to be in the direction of 
higher validities for the original test scores. There are several possible explanations for this unexpected 
finding which will be addressed in the General Discussion section. However, the present results 
suggest that the concurrency effects were small, if they existed at all. 
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CORRECTIONS FOR RESTRICTIONS OF RANGE 



Description of the Problem 

All of the analyses reported thus far in this paper were based upon data collected from 
samples of airmen in Basic Military Training. Individuals in these samples were accepted into the U.S. 
Air Force from a general applicant population only after having met certain minimum aptitude 
requirement levels on ASVAB Aptitude Indexes (ATs). This selection process resulted in a restriction 
of range in the ASVAB test score distributions, which, in turn, yielded lower ASVAB test validities 
than would have been obtained in the full applicant population. There was also a restriction of range in 
the LAMP test-score distributions, to the extent that the LAMP tests correlated with the selection 
criteria. Finally, the differential restrictions of range due to selection resulted in overestimations of 
what the unique contribution of LAMP tests to ASVAB tests would be in the applicant population. 



A decision was made to correct all of the intercorrelations and validities for range restrictions. 
However, a question arose. Which of three populations should serve as the basis for such corrections? 
The most stable would be the 18-23 year-old youth population for 1980, which was used for establishing 
the ASVAB norms. However, in my judgment, use of this population would not provide the most 
meaningful indication of the utility of the ASVAB in making selection decisions for an all-volunteer 
force, nor would it provide the most meaningful indication of the potential contribution of the LAMP 
tests to the ASVAB tests in making such decisions. The value of mental tests in maintaining an all- 
volunteer force should be gauged in terms of their power to identify the best-qualified individuals from 
among those being considered for selection. This being the case, the value of tests to the Department 
of Defense should be gauged in terms of their ability to identify the best qualified from among those 
applying for entry into the Uniformed Services; while the value of the tests to the U.S. Air Force should 
be gauged in terms of their ability to identify the best qualified individuals from among those applying 
for entry into the U.S. Air Force. 

It is recognized that the nature of applicant populations changes over time; therefore, they 
would not be appropriate for establishing operational test norms. However, they can be used to 
estimate the value of the tests in selecting the best qualified from those being considered across a given 
time period. In the present study, a method suggested by Mifflin & Verna (1977) was applied to 
correct the test validities and intercorrelations to a random sample of 10,853 USAF individuals who 
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applied for entry into the Air Force during 1980-1989. Results from analyses of these data are judged 
to be the best indication of the contribution of LAMP tests to ASVAB tests in making selection 
decisions in an all-volunteer force environment. Corrections would have been made to the DOD 
applicant population during this same time-period, but unfortunately, data were not available when this 
report was being prepared. 

Data were available for approximately 10,000 subjects designed to be representative of the 
25,408,193-case 1980 youth population described by Maier & Sims (1986). Corrections were made to 
this sample to get a rough indication of test utility in case of an all-out draft. However, in my 
judgment, use of this sample leads to an overcorrection. In the event of an all-out draft, a portion of 
those individuals in the upper-aptitude ranges would be siphoned off into officer training programs, 
and some in critical jobs or in college might be given deferments. Undoubtedly there would also be 
some individuals who are physically or mentally handicapped, or who have criminal records, who would 
be rejected prior to psychological testing. These factors lead one to predict that the range of aptitudes 
in the sample considered for draft into the enlisted ranks would be less than the range of aptitudes in 
the 1980 youth population. In spite of these concerns, corrections to this population were made. 

Results 

Tables 23 reports validities of the ASVAB and LAMP tests for the Study 1 gates course 
accuracy and latency criteria in the computing sample, as well as those corrected to the USAF 
applicant sample and the 1980 youth sample. Table 24 reports the same information based on Study 2 
data. Table 25 reports data relating to the contribution of LAMP tests to ASVAB tests in predicting 
Study 1 gates course accuracy and latency criteria in the USAF applicant sample, while Table 26 
reports these estimated contributions in the 1980 youth sample. Tables 27 and 28 are comparable to 
Tables 25 and 26, except that they are based upon Study 2 data. 

Table 29 presents a summary of LAMP test contributions to ASVAB in accounting for Study 1 
and Study 2 accuracy and latency criterion variance in the computing sample, in the USAF applicant 
sample, and in the 1980 youth sample. These contributions are impressive, even when the estimates 
are made using differences in adjusted R 2 s which have been corrected to the 1980 youth sample. 
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Table 23. Validities of ASVAB and LAMP Tests for Study 1 Accuracy and Latency Criteria in the 
Computing Sample, the Air Force Applicant Sample and the 1980 Youth Sample 



Test 




ACCURACY 






LATENCY 




CS 


AAS 


YS 


CS 


AAS 


YS 


ASVAB 














GS 


.13 


36 


32 


-.01 


-.13 


-.22 


AR 


.40 


.54 


.65 


-.06 


-.16 


-.25 


WK 


.20 


39 


.54 


-.03 


-.14 


-.24 


PC 


.16 


.38 


.51 


-.03 


-.14 


-.24 


NO 


.08 


.24 


.44 


-.13 


-.21 


-.29 


CS 


.09 


.22 


.40 


-.10 


-.17 


-.25 


AS 


.10 


.28 


39 


.01 


-.05 


-.13 


MK 


.40 


.51 


.63 


-.06 


-.15 


-.24 


MC 


.29 


.43 


.53 


.03 


-.06 


-.15 


EI 


.26 


.38 


53 


.05 


-.03 


-.14 


LAMP 














CRA 


.27 


.29 


.32 


.05 


.03 


-.00 


NIA 


.29 


.32 


.36 


.07 


.04 


-.01 


PIA 


.27 


.27 


.28 


.08 


.07 


.05 


DSA 


.32 


.41 


.50 


-.07 


-.12 


-.19 


NFA 


.19 


.23 


.28 


.04 


.01 


-.02 


NRA 


.47 


.57 


.67 


-.07 


-.15 


-.24 


MIA 


.35 


.44 


.53 


.14 


.07 


-.02 


ABA 


.44 


.55 


.65 


.03 


-.08 • 


-.18 


NAA 


.48 


.58 


.68 


-.11 


-.19 


-.27 


CRL 


.05 


.01 


-.07 


.14 


.16 


.19 


PIL 


.11 


.03 


-.09 


.32 


.35 


.38 


NFL 


.07 


-.04 


-.21 


.31 


.34 


.40 


MIL 


.07 


-.09 


-.26 


.21 


.26 


.32 


ABL 


.34 


.25 


.16 


.35 


.37 


.39 


NIL 


.09 


-.02 


-.17 


.36 


.39 


.43 



Notes. Test abbreviations defined in Figure 1. CS = Computing Sample; AAS = Air Force 
Applicant Sample; YS= 1980 Youth Sample. 
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Table 24. Validities of ASVAB and LAMP Tests for Study 2 Accuracy and Latency Criteria in the 
Computing Sample, the Air Force Applicant Sample, and the 1980 Youth Sample 



ACCURACY LATENCY 



Tuff 

l esi 






YS 


CS 


AAS 


YS 


ASVAB 














GS 


21 


.42 


57 


-.19 


-31 


-.40 


AR 


34 


.53 


.65 


-.18 


-.29 


-39 


WK 


.17 


.44 


.59 


-.06 


-.22 


-33 


PC 


.12 


s-) {-\ 

39 


.52 




91 

-.Z-JL 




[NU 


.ID 


2Q 


49 


-.12 


-.19 


-31 


Co 


1 1 
.11 




42 


- 11 


-.18 


-.29 


AS 


All 




42 


-.02 


-.14 


-.23 










- 17 


-.26 


-36 


ML 






S7 


- 08 


-.20 


-.29 


EI 


.lo 




<< 


-05 


-.16 


-.29 


LAMP 














GKA 


.17 


.40 


.55 


-.21 


-.32 


-.40 


FRA 


36 


.43 


.48 


-.15 


-.20 


-.25 


MCA 


.40 


.51 


.60 


-.13 


-.21 


-.30 


N1A 


.23 


.28 


.36 


-.02 


-.05 


-.12 


PIA 


.24 


.25 


.29 


.04 


.02 


-.02 


CRA 


.28 


31 


.34 


.07 


.04 


-.01 


ABA 


.44 


.58 


.68 


-.07 


-.18 


-.29 


WAA 


39 


.53 


.62 


-.14 


-.24 


-.33 


NAA 


.46 


56 


.65 


-.14 


-.22 


-31 


GKL 


-.15 


-.39 


-.55 


.27 


.36 


.43 


NIL 


-.11 


-.22 


-.37 


.37 


.41 


.47 


PIL 


-.08 


-.19 


-.32 


.38 


.41 


.47 


ABL 


.30 


.23 


.15 


.29 


.30 


.32 


CRL 


-.17 


-.25 


-.36 


.28 


.32 


37 



Notes. Test abbreviations defined in Figure 1. CS = Computing Sample; AAS = Air Force 
Applicant Sample; YS = 1980 Youth Sample. 
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Table 25. Contribution of LAMP Tests to ASVAB Tests in Accounting for Study 1 Criterion Variance 

in the USAF Applicant Sample 



R Chg. 



P value 



A pproach 1 



ASVAB to LAMP 
LAMP to ASVAB 



.035 
.198 



Overall R = .743 R 2 = .552 Adj R 2 = .525 



ACCURACY 



LATENCY 



3.319 
13.412 



.000 
.000 



ASVAB to LAMP 
LAMP to ASVAB 



.005 
.211 



.316 
8.129 



.977 
.000 



Overall R = .525 R 2 = .271 Adj R 2 = .228 



A pproach 2 



ASVAB 



ACCURACY 
LAMP LAMP + ASVAB 



L TO A 



R 

Adj. R 2 



.595 
.354 
.339 



.719 
.516 
.500 



.723 
.552 
.525 



.198 
.186 



LATENCY 



AdjR 



.245 
.060 
.039 



.515 
.265 
.240 



.521 
.271 
.228 



.211 
.189 



Notes. L to A reports the estimated contribution of LAMP tests to ASVAB tests. 
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Table 26. Contribution of LAMP Tests to ASVAB Tests in Accounting for Study 1 Criterion Variance 

in the 1980 Youth Sample 



R 2 Chg. 



P value 



A pproach 1 



ACCURACY 



ASVAB to LAMP 
LAMP to ASVAB 



.029 
.160 



3.411 
12.411 



.000 
.000 



Overall R=.798 R 2 =.637 AdjR 2 = .616 



LATENCY 



ASVAB to LAMP 
LAMP to ASVAB 



.005 
.201 



.330 
8.129 



.973 
.000 



Overall R = .551 R 2 = 304 Adj R 2 = .263 



R 

Adj. R 2 



ASVAB 

.619 
.477 
.465 



ACCURACY 
LAMP LAMP + ASVAB 



.780 
.608 
.594 



.798 
.637 
.616 



L TO A 

.160 
.151 



R 

AdjR 2 



.320 
.103 
.082 



LATENCY 



.546 
.298 
.274 



.551 
.304 
.263 



.201 
.181 



Notes. L to A reports the estimated contribution of LAMP tests to ASVAB tests. 
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Table 27. Contribution of LAMP Tests to ASVAB Tests in Accounting for Study 2 Criterion Variance 

in the Air For ce Applicant Samp le 

R^Chg. E P value 

Approach 1 



ACCURACY 

ASVAB to LAMP .031 2.831 .002 

LAMP to ASVAB .176 11.395 .000 

Overall R = .743 R 2 = .552 Adj R 2 = .526 



LATENCY 



ASVAB to LAMP 
LAMP to ASVAB 



.026 
.192 



1.584 
8.260 



Overall R = .570 R 2 = .325 Adj R 2 = .285 



Approach 2 



.109 
.000 



Adj. R 



s 



ASVAB 

.613 
.376 
.361 



ACCURACY 
LAMP LAMP + ASVAB 



.722 
.521 
.505 



.743 
.552 
.526 



LTOA 

.176 
.165 



AdjR 



.364 
.133 
.112 



.547 
.299 
.275 



LATENCY 



.570 
.325 
.285 



.192 
.173 



Notes. L to A reports the estimated contribution of LAMP tests to ASVAB tests. 
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Table 28. Contribution of LAMP Tests to ASVAB Tests in Accounting for Study 2 Criterion Variance 
in the 1980 Youth Sample 

R 2 Chg. F P value 

Approach 1 



ACCURACY 



ASVAB to LAMP 
LAMP to ASVAB 



.027 
.139 



3.054 
11.396 



.002 
.000 



Overall R = .805 R 2 = .647 Adj R 2 = .627 



LATENCY 



ASVAB to LAMP 
LAMP to ASVAB 



.028 
.178 



1.799 
8.255 



.059 
.000 



Overall R = .611 R = 374 Adj R" = .337 



A pproach 2 



Adj. R 



ASVAB 

.713 
.508 
.497 



ACCURACY 
LAMP LAMP + ASVAB 



.788 
.621 
.608 



.805 
.647 
.627 



LTOA 

.139 
.130 



Adj R 



.442 
.196 
.176 



.588 
.346 
.324 



LATENCY 



.611 
.374 
.337 



.178 
.161 



Notes. L to A reports the estimated contribution of LAMP tests to ASVAB tests. 
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Table 29. Summary of LAMP Test Contributions to ASVAB in Accounting for Study 1 and Study 2 



Accuracy and Latency Criterion Variance 



Approach 1 














ACCURACY 




LATENCY 




Stvd Y l 


Studv2 


Studvl 


Studv2 


Computing Sample 


23.6% 


22.2% 


22.0% 


20.7% 


A.F. Applicants 


19.8% 


17.6% 


21.1% 


19.2% 


Youth Samnle 


16.0% 


13.9% 


20.1% 


17.8% 


Approach 2 














ACCURACY 




LATENCY 




Studvl 


Study2 


Studyl 


Studv2 


Computing Sample 


22.0% 


20.6% 


19.7% 


18.5% 


A.F. Applicants 


18.6% 


18.9% 


165% 


173% 


Youth Sample 


15.1% 


18.1% 


13.0% 


16.1% 



Notes. All values are in terms of the percent valid variance added by the LAMP tests to 
ASVAB tests in accounting for criterion variance. 



While the difference in R s for the full and restricted model is the normal way of defining the 
unique contribution of one set of tests to another in accounting for criterion variance, Brogden (1946) 
argues that variation in the efficiency of a selective or predictive instrument (either a test or test 
composite) is a direct linear function of variation of its product-moment correlation (or multiple 
correlation) with the predicted variable. He demonstrates that the correlation (or multiple correlation) 
coefficient directly indicates the proportion of maximum saving that is actually obtained with use of the 
selection instrument, where the maximum saving is that obtained by selecting on the criterion. Data in 
Table 30 are provided to reflect the predictive efficiency of ASVAB and LAMP tests in accordance 
with this view. The values in the first two columns of numbers are the multiple correlation coefficients 
for the ASVAB and LAMP tests for the various gates course criteria obtained in the computing 
sample, in the USAF applicant sample, and in the 1980 youth sample. The values in the third column 
are the multiple correlations for the combined ASVAB and LAMP tests for these same criteria. The 
last column reports the differences in full model multiple correlation coefficients and the ASVAB 
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multiple correlation coefficients, which, according to Brogden would indicate the increase in predictive 
efficiency contributed by the LAMP tests. 

Table 30. Summary of Increases in Efficiency Contributed by LAMP tests to ASVAB Tests in the 
Prediction of Accuracy and Latency Criteria* 

ACCURACY, STUDY 1 

Increased 

Predictive Efficiency Predictive 
ASVAB LAMP A ± L Efficiency 

Computing Sample .478 .656 

A.F. Applicants .595 .719 

Youth Sample .691 .780 



.682 .204 
.743 .148 
.798 .098 



ACCURACY, STUDY 2 

Computing Sample .460 .635 .659 .199 

A.F. Applicants .613 .722 .743 .130 

Youth Sample .713 .788 .805 .092 



LATENCY, STUDY 1 

Computing Sample .161 .490 .496 .335 

A.F. Applicants .245 .515 .521 .276 

Youth Sample 320 .546 .551 .231 



LATENCY, STUDY 2 

Computing Sample .267 

A.F. Applicants 364 

Youth Sample .442 

Notes. Predictive efficiency is defined as the expected value of the gain effected by selecting with the 
selection composite to the gain that would be effected by selecting on the criterion itself. See the text 
for addition comments. 



.507 .526 .259 

.547 .570 .206 

588 .611 .169 



GENERAL DISCUSSION 



The present paper is atypical of those usually published in the LAMP series in that it reports 
the results of applied rather than basic research. It contributes little if anything to our understanding of 
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how individuals think, remember, solve problems, or process information in acquiring new knowledge 
or skills. However, basic research on cognition supported by the Air Force should have some 
probability of ultimate payoff in terms of improving selection, assignment, training, or some other 
aspect of the personnel system. Applied studies need to be conducted on occasion to see if progress is 
being made toward one or more of these goals. The levels of incremental validities found here at the 
very least attest to the fact that the LAMP approach is promising. However a number of factors should 
be considered in evaluating the final results, some of which are briefly treated in the following 
paragraphs. 

Nature of the Learning Task 

The learning task used in the present study was meaningful, but it was somewhat unusual in 
the sense that the airmen used as subjects had little or no incoming knowledge specific to the materials 
being taught. That is, it represented a novel learning situation. This is one of the reasons why AS VAB 
measures of technical and verbal knowledge, which have substantial validity for many technical school 
grades, had such low validities in the present study. It is anticipated that LAMP tests will add 
considerably less valid variance to the ASVAB tests in predicting technical school training grades than 
they did in predicting the criteria used in the present study. Most technical school grades reflect 
academic learning, and ASVAB tests historically have done well in predicting individual differences in 
academic learning. However, it is anticipated that the value of LAMP tests will be revealed against 
criteria involving the development of procedural skills, which tends to take place in the job 
environment. 

Concurrency Effects 

No concurrency effects were found in the present study. This was somewhat of a surprise, 
since one would expect abilities measured at the time of entry into a training course to have higher 
validities than abilities measured at some prior point in time. To the extent that this expectation was 
the case, the LAMP tests would have had an advantage over the ASVAB tests, which were 
administered some months earlier. Some possible explanations for the failure to find concurrency 
effects are given below: 

1) It's simply a matter of ASVAB forms. Several forms of the ASVAB were administered in 
the operational testing stations, whereas a single form was used in the retesting session. Perhaps, for 
some unknown reason, the readministration form was inferior to the forms used in the operational 
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testing stations. This would be a simple explanation, but one which is difficult to accept by anyone who 
knows the care taken in ASVAB construction and norming. 

2) The subjects were not motivated to do their best on the ASVAB retest, but regained 
motivation when faced with the novel computerized tests in the LAMP battery. 

3) The ASVAB retest scores were poorer measures of working memory than were the 
original test scores. It is possible that there was differential forgetting of procedural and declarative 
knowledge between test and retest. If retention abilities do not predict the acquisition of a novel skill, 
the addition of retention factors to the concurrent ASVAB would lower its validity. 

4) There is little or no concurrency effect since abilities do not change substantially during the 
interval between test and retest. This is the third study to suggest that currency effects are small. 
Perhaps they are in fact of little consequence. 

Common Methods 

There is a question of how much of the contribution of LAMP tests to ASVAB tests can be 
attributed to the use of common methods. It is true that the LAMP tests and the logic gates courses 
were both administered by microcomputer, while the ASVAB tests were administered by paper and 
pencil. 

It would be difficult to accept that common methods played a significant role in determining 
the relationships of LAMP tests with accuracy criteria. If an individual correctly analyzed the input 
signals to a logic gate and knew the output to be either high or low, it should not have made much 
difference whether the answer was recorded by marking an answer sheet or pressing a computer key. 

The same cannot be said with respect to predicting latency criteria. In the computing sample, 
none of the ASVAB test scores were significantly related to latency scores from the logic gates courses, 
while all of the latency scores from the LAMP tests showed validity for the course latency data. A 
basic question is "How much of an unfair advantage did the use of common methods give to the LAMP 
tests? H . 

To answer this question, one first should decompose the criterion variance into two parts: 1) 
that part which was relevant to the skill being taught, and 2) that part which was not relevant to the skill 
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being taught. The relevant variance in the logic gates course latency data was associated with the time 
required for subjects to mentally process information to determine gate output signals. The non- 
relevant variance was associated with the time required to record answers using the computer 
keyboard. One might argue, then, that the variance associated with common methods may have given 
the LAMP tests some unfair advantage with respect to answer entry— but not with respect to 
measuring the time required for mental processing. 6 



Processing Accuracy 

The LAMP measures of processing accuracy made a small but significant contribution to other 
LAMP measures in predicting accuracy criteria from the gates courses. These scores were extracted 
from simple processing speed tests having high accuracy levels. Within testing sessions, individuals who 
made errors on one of these tests tended to be the same individuals who made errors on the others. 
However, it is important to demonstrate that such tendencies are stable across time. If they are not, 
then the present analyses may have capitalized on some common-factor variance between the LAMP 
tests and the logic gate course criteria which will not hold up in a predictive sense. 



Testing Time S 

Testing time is important when one is considering the utility of new tests for possible inclusion 
in future operational bat:eries. It is reasonable to ask whether the apparent advantage of LAMP tests 
in the present study was at the expense of excessive testing time. Unfortunately, it is not possible to 
give precise time requirements for the LAMP tests or for the logic gates courses, since the subjects 
proceeded at their own rate. However, the question can be answered in a gross fashion. About one 
hour was required for the slowest individuals to complete the course in Study 1, and about one-and- 
one-half hours was required for the slowest individuals to complete the course in Study 2. Since only 
three hours of testing time was available, a gross estimate is that the LAMP tests took about two hours 
for administration in Study 1 and one-and-one-half hours for administration in Study 2. In each 
instance, this is less time than was required for administration of the AS VAB. 
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6 One of the weaknesses of paper-and-pencil tests is that they do not provide a means for accurately 
measuring the speed of mental processing. In the case of solving logic gate problems, precision is 
needed to a tenth of a second in order to reliably discriminate between subjects. The wave of the 
future is computer-administered courses and computerized job tasks. The day may come, when even 
the time associated with keyboard entry is course and/or job related. 
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Of course, it should be recognized that neither the ASVAB nor the LAMP tests were designed 
with the lope gates course in mind. Both test sets were designed to measure knowledge and abilities 
relevant to learning in general. It is interesting to note that one of the LAMP tests with the highest 
validity for the accuracy criteria was Grammatical Reasoning, which took only five minutes for 
administration. 



Other Considerations 

Two factors stand out as explaining individual differences in the acquisition of declarative 
knowledge about gate rules and the development of procedural skill in solving logic gate problems. 
Information processing speed and working memory capacity were the best predictors of procedural 
skill development, and working memory capacity alone was by far the best predictor of rule learning. 

The fact that ASVAB does not contain good measures of processing speed accounts for its 
inability to predict procedural skill development in the present study. The ASVAB does contain three 
tests which loaded on the Working Memory factor, and they accounted for essentially all of the 
ASVAB validity for gates course accuracy criteria. However, the ASVAB measures of working 
memory are heavily dependent on declarative and procedural knowledge gained in academic schooling, 
such as taking a square root or setting up equations to solve for an unknown. On the other hand, the 
LAMP tests of working memory yielded higher validities for the gates course accuracy criteria, and are 
less dependent on academic knowledge. For example, in the computing sample , the Arithmetic 
Reasoning and Mathematics Knowledge test validities for the average accuracy scores across all blocks 
in Study 2 were .34 and 38, respectively, while the Grammatical Reasoning, and Numerical 
Assignment tests in the LAMP battery correlated .44, and .46 with this same criterion. This advantage 
was maintained even in the data corrected to the USAF Applicant sample, where the values were .53 
and .54 for the two ASVAB tests, compared with .58 and .56 for the two LAMP tests. 

An important side benefit of using processing-based predictors is that they are not only more 
valid against certain criteria, but they also tend to be less related to Race and Sex. For example, in the 
Study 2 computing sample, the Arithmetic Reasoning test correlated .22 with Race (1/0= Non- 
Black/Black) while the Mechanical Comprehension test correlated .33 with Race and .29 with Sex 
(l/0=Male/Female). In contrast, the Grammatical Reasoning test (the highly valid 5-minute test), 
correlated only .11 with Race and -.01 with Sex. There appears to be some merit in considering some 
type of processing-based working memory test for future forms of the ASVAB. 
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CONCLUSIONS 



Table 29 presents a summary of the major findings from Studies 1 and 2. In general, the 
LAMP tests added about 20% unique valid variance to the ASVAB in predicting both accuracy and 
latency block averages in the two logic gate courses. When the data were corrected for restrictions in 
range to the USAF applicant sample, the contributions of LAMP tests to ASVAB tests still ranged 
from 17% - 20% . The data in Table 30 show substantial contributions of the LAMP tests to the 
ASVAB tests in terms of increased predictive effeciency, where predictive efficiency is defined as the 
expected value of the gain effected by selecting with the selection composite to the gain that would be 
effected by selecting on the criterion itself. 

Analyses indicated that the advantage of the LAMP tests over the ASVAB tests was not the 
result of a concurrency effect. Nor was it obtained at the expense of excessive testing time. Although 
the LAMP tests and criterion tests used common methods, arguments were presented contending that 
most of the covariance associated with common methods was meaningful to the task being learned and 
performed. 

While the findings in the current study were for learning and skill development in a single 
subject-matter area, this subject-matter was selected because of its relevance to meaningful USAF 
functions. The results are viewed as being highly encouraging with respect to the potential contribution 
of the Learning Abilities Measurement Program (LAMP) to the USAF personnel system. 



ERLC 



53 61 



REFERENCES 



Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: MIT press. 

Baddeley, A. D. (1968). A 3-rain reasoning test based on grammatical transformation. Psychonomic 
Science, 10, 341-342. 

Brogden, H. E. (1946). On the interpretation of the correlation coefficient as a measure of predictive 
efficiency. The Journal of Educational Psychology, 37, 65-76. 

Carlson, R. A., Khoo, H. ¥L, Yaure, R. G., & Schneider, W. (1990). Acquisition of a problem-solving 
skill: Levels of organization and use of working memory Journal of Experimental Psyhology: General, 
119, 193-214. 

Christal, R.E. (1988). [Correlation matricies and factor analyses results from four working memory 
test batteries]. Unpublished raw data. 

Christal, R.E. (1989). Estimating the contribution of experimental tests to the Armed Services Vocational 
Aptitude Battery (AFHRL-TP-89-30, AD-A216 122). Brooks AFB, TX: Manpower and Personnel 
Division, Air Force Human Resources Laboratory. 

Department of Defense (1984). Test maitual for the Armed Services Vocational Aptitude Battery (DOD, 
1340.12AA). North Chicago, IL: U. S. Military Entrance Processing Command. 

Divgi, D. R. (1990). Concurrent versus enlistment ASVAB in evaluation of new tests (CMR 89-234). 
Alexandria, VA; Center for Naval Analyses. 

Gitomer, D. H. (1984). A cogtxitive analysis of a complex troubleshooting task. Unpublished doctoral 
dissertation. University of Pittsburg, Pittsburg, PA. 

Kyllonen, P. C. (1985a). Dimension of information processing speed. (AFHRL-TP-84-56, AD- A- 154-778). 
Brooks AFB, TX: Manpower and Personnel Division, Air Force Human Resources Laboratory. 

Kyllonen, P. C. (1985b). Individual differences in learning and cognition: Tlxe four-source proposal. 
(AFHRL-TR-85-17, AD-B096 160L). Brooks AFB, TX: Manpower and Personnel Division, 
Air Force Human Resources Laboratory. 



62 



Kyllonen, P. C. & Christal, R. E. (1989). Cognitive modeling of learning abilities: A status report of 
LAMP. In R. Dillon & J. W. Pellgrino (Eds.), Testing: Theoretical and applied issues. 
New York: Freeman (also [1988] Tech. Paper No. AFHRL-TP-87-66, Brooks AFB, TX: Manpower 
and Personnel Division, Air Force Human Resources Laboratory). 

Kyllonen, P. C, & Christal, R. E. (1990). Reasoning is (little more than) working memory capacity. 
Intelligence, 14. 389-433.. 

Kyllonen, P. C, Tiirre, W. C, & Christal, R. E. (1988). Knowledge and processing speed as determinants 
of associative learning (AFHRL-TP-87-68, AD-A207 106). Brooks AFB, TX: Manpower and 
Personnel Division, Air Force Human Resources Laboratory. 

Maier, M. H. & Sims, W. H. (1986). The ASVAB score scales: 1980 and World War II. (CNR 116) 
Alexandria, VA: Marine Corps Operations Group, Center for Naval Analysis. 

Mifflin, T. L. & Verna, S. M. (1977). A method to correct correlation coefficients for the effects of multiple 
curtailment. (CRC 336). Arlington, VA: Marine Corps Operations Analysis Group, Center for Naval 
Analysis. 

Shute, V. J. (in press-a). The role of aptitude-treatment interactions in cognitive skill diagnosis. To 
appear in J. W. Regian & V. J. Shute (Eds.). Cognitive diagnosis and automated instruction. Hillsdale, 
NJ: Lawrence Erlbaum Associates. 

Shute, V. J. (in press-b). Who is likely to acquire programming skills? to appear in the Journal of 
Educational Computing Research, 7, 1-. 

Shute, V. J. & Glaser, R. (1990). Large-scale evaluation of an intelligent tutoring system: Smithtown. 
Interactive Learning Environments, J, 51-76. 

Shute, V. J. & Pena, C. M (1990). Acquisition of programming skills. (AFHRL-TP-89-54). Brooks AFB, 
TX: Manpower and Personnel Division, Air Force Human Resources Laboratory, 



ERLC 



55 63 



Thurstone, L.L., & Thurstone, T.G. (1941). Factorial studies of intelligence. Psychometric 
Monographs, 2. 

Woltz, D.J. (1988). An investigation of the role of working memory in procedural skill acquisition. 
Journal of Experimental Psychology: General, 117, 319-331. 



APPENDIX A 
DESCRIPTIONS OF ASVAB AND LAMP TESTS 



The first ten tests listed are from the ASVAB and their descriptions are based on information 
in the Department of Defense Bulletin DOD 1340.12AA (1984). 

ASVAB Tests Used in Study 1 

General Science . This test contains 35 standard vocabulary items, such as "Which of the 
following foods contain the most iron? (a) eggs, (b) liver, (c) candy, (d) cucumber." 

Arithmetic Reasoning. This test consists of 30 arithmetic word problems, such as the 
following: "Pat put in a total of 16 1/2 hours on a job during 5 days of the past week. How long is Pat's 
average workday? (a) 3 hours, (b) 3 hours, 15 minutes, (c) 3 hours, 18 minutes, (d) 3 hours, 25 
minutes." 

Word Knowledge. This test consists of 35 standard vocabulary items, such as "The wind is 
variable today, (a) mild, (b) steady, (c) shifting, or (d) chilling." 

Paragraph Comprehension. This test consists of 15 paragraphs, each l-to-3 sentences long, 
followed by a multiple-choice question of the paragraph's content. An example is the following: 

From a building designer's standpoint, three things that make a home livable are the client, the 
building site, and the amount of money the client has to spend. According to the passage, to make a 
home livable 

(a) it can be built on any piece of land; 

(b) the design must fit the designer's income; 

(c) the d^agft-must fit the owner's income and site; 

(d) the prospective piece of land makes little difference. 

Numerical Operations. This is a speeded test, consisting of 50 number-fact items (e.g., 2x6 
= ? (a) 4, (b) 8, (c) 3, (d) 12). Examinees are given 10 minutes to finish as many of the items as they 
can. 
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Coding Speed. This is a speeded test consisting of 84 items designed to measure how quickly 
an examinee can find a number in a table. An item consists of a word followed by five 4-digit number 
strings (e.g., (a) 6456, (b) 7150, (c) 8930, (d) 9645). The subject's task is to (a) look up the word's 
number code in a key consisting of 10 word-code pairs placed on top of the page, then (b) select the 
letter associated with that number code. Subjects are given 10 minutes to complete as many items as 
they can. 

Auto and Shop Information. This tests consists of 25 questions about automobiles, shop 
practices, and the use of tools. An example question is: "A chisel is used for (a) prying, (b) cutting, (c) 
twisting, (d) grinding." Some of the questions include drawings of common tools or parts. 

Mathematics Knowledge . This test consists of 25 arithmetic word problems (primarily 
algebra, but also area, square root, percentage, and simple geometry), such as the following: "If 3X = 
5, then X = : (a) -2, (b) -5/3, (c) 3/5, (d) -3/5." 

Mechanical Comprehension. This test consists of 25 questions relating to general mechanical 
and physical principles. The questions are normally accompanied by drawings. For example, a drawing 
may include three meshed gears with an arrow indicating the direction of rotation of one of the gears. 
The question might be "Which of the other gears is moving in the same direction as gear 1? (a) Gear 
2, (b) Gear 3, (c) Both of the other gears, (d) Neither of the other gears." 

Electrical Information. This test consists of 20 questions relating to knowledge of electrical, 
radio, and electronics information. Some of the questions contain drawings, while others do not. The 
following is an example question: "Which of the following has the least resistance? 
(a) Iron, (b) Wood, (c) Silver, (d) Rubber." 



LAMP Tests Included in Study 1 

Choice Reaction Time. This test consists of 5 blocks of 20 items, for a total of 100. In each 
item, a block of 4 asterisks is presented on the right center or left center of the screen. If the asterisks 
appear on the right center, the subject is requested to press the key. If they are on the left center, 
the subject is requested to press the 'D' key. Latency in milliseconds and 'CORRECT or 'WRONG' 
feedback is provided after each response. There is a 1-second intertrial time between items. Summary 
feedback is provided after each of the five blocks. Although individuals are requested to respond as 
quickly as possible without making mistakes, mistakes arc in fact made by most subjects. The primary 
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score extracted from the response data is 'median latency* in milliseconds, 
also computed and is used in the analyses. 



A 'percent correct' score is 



Physical Identity . This test consists of 5 blocks of 20 items each, for a total of 100 items. In 
each item, two words are presented on the screen, one above the other. The words are either both in 
upper-case letters or both in lower-case letters. In 50 percent of the trials, the two words are physically 
identical. In the other 50 percent of the trials, the words are different, even though the case for the two 
words is the same. There is a warning asterisk prior to displaying each item. The subject is requested 
to press the 'L' key if the two words are physically identical, or the'D' key if they are not. Response 
latency feedback is provided after each correct response, and the word 'WRONG' is presented, along 
with the sound of a buzzer, after each incorrect response. Summary feedback is provided at the end of 
each block. As in the case of the Choice Reaction Time test, both median latency and accuracy scores 
are obtained from the test results. 

Name Identity. As in the case of the Choice Reaction Time and Physical Identity tests, this 
test contains 5 blocks of 20 items each, for a total of 100 items. Two words are presented in the middle 
of the screen, one above the other. One of the words is presented in upper-case letters, while the other 
is presented in lower-case letters. In 50 percent of the items, the two words are the same; in the other 
50 percent, the two words are different (have a different name). The subject is requested to press the 
'L' key if the two ^vords have the same name, or the 'D', key if they have different names. The display 
format and the feedback provided is the same as in the Physical Identity test, above. Both latency and 
accuracy scores are derived from the test. 

Meaning Identity. Meaning Identity is another information processing speed measure which 
contains 5 blocks of 20 items each. In each item, two words are displayed, one above the other, in the 
center of the screen immediately after a warning asterisk. Both words are presented in lower-case 
letters. The subject is instructed to press the 'L' key if the words have a similar meaning (are 
synonyms), or the 'D' key if they do not have similar a similar meaning. The words were selected to 
minimize the role of knowledge, since the goal is to measure speed of memory search, memory 
retrieval, and feature comparison. As in the case of the other processing speed measures, latency 
feedback is given on correct responses and 'WRONG* feedback is given on incorrect responses. Both 
median latency and percent correct scores were computed for all subjects. 

Number Fact Retrieval. Subjects are given four sets of simple arithmetic problems: 50 
addition (4 + 2), 50 multiplication (3 X 2), 25 subtraction (8 - 4), and 25 division (9 / 3). In each 
instance, the problem is displayed in the middle of the screen, with a proposed answer centered below 
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it. The subject is instructed to press the 4 L' key if the proposed answer is correct, or the 'D' key if the 
proposed answer is incorrect. Each problem is preceded with a warning star. Median latency and 
percent correct scores are obtained for each subject. 
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AB Grammatical Reasoning, AB Grammatical Reasoning is a computerized version of 
Baddele/s (1968) "3 minute test of grammatical reasoning." Subjects determine whether a sentence 
such as "A follows B M is consistent with an arrangement of the letters A and B (either "AB M or "HA"). 
The sentence and the letters appear on the screen simultaneously. Sentences are varied in relation 
(precedes vs. follows), voice (active vs passive), negation (present vs. passive), subject (A vs. B), and 
truth (true vs. false). There are 2 = 32 X 2 (replications) = 64 items, divided into four blocks. The 
computer provides latency and 'CORRECT or 'WRONG' feedback after each item, and summary 
information at the end of each block. The computer displays items one-at-a-time, with no time limit. 
Both median latency and percent correct scores are computed for each subject. 

123 Number Reduction. The 123 Number Reduction test is a modification of Thurstone and 
Thurstone's (1941) ABC Test. Subjects are asked to translate 2-, 3-, 4- ,5- and 6-digit numbers 
according to two rules. Rule 1 is to rewrite two successive same digits (e.g., 2 2) to that digit (i.e., 2). 
Rule 2 is to rewrite two successive different digits (e.g., 3 2) to the third digit (i.e., 1). The subject is 
instructed to evaluate pairs ieft-to-right and to enter the final digit resulting from the complete 
recoding sequence. This test was designed to measure working memory, since it involves processing in 
the presence of a memory load. The test is scored fo: percent correct only. 

Digit Span. Digit Span is another test designed to measure working memory. The computer 
displays 5, 7, or 9 digits successively for 1 second each, left-to-right, 2 characters apart on the display 
screen. Immediately after the presentation, probe digits are presented one at-a-time from either the 
beginning or end of the list. The subject's task is to indicate (by pressing the 'L' or 'D' key) whether 
the probe matches the displayed digit. After the subject responds, a probe for the next-to-last digit is 
presented and subjects respond in the same fashion. On half of the items, probes are from the end of 
the list, and on the other half, from the beginning of the list. From 2 to 5 probes are displayed for each 
list. Subjects are given credit for each of the probes they correctly discriminate. Items are blocked by 
digit strings and are displayed in order (5-digit items first, followed by 7-digit and then two blocks of 9- 
digit items). There are eight 5-digit items (21 responses), ten 7-digit items (34 responses), 5 items in 
the first 9-digit block (21 responses), and 5 items in the second 9-digit block (23 responses), for a total 
of 4 blocks. The test provides summary percent correct feedback at the end of each block. It is scored 
only for percent correct. 
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ABC Numerical Assignment, This task, which in previous studies has been found to be a 
strong measure of working memory, consists of 21 items, each requiring three responses (63 probes, 
total), presented in a single block. Each item presents a sequence of screens in which each of the 
letters A, B, and C are assigned values or equations. A typical (but difficult) item is: (Screen 1) "A » B 
/2"; (Screen 2) "B - C-4)"; (Screen 3) "C = 8"; (Screen 4) "B = ?"; (Screen 5) "A - ?"; (Screen 6) 
"C = ?". Letters were assigned either values (e.g., A = 5) or equations (e.g., A = B/2). In equation 
assignments, it was sometimes possible to solve for the equation with information already known. In 
other cases (such as in the example item), it is necessary to hold the equation in memory and wait for 
later screens to provide missing information that make it possible to solve the equation. The test 
allows unlimited per-screen study time, but subjects are not able to scroll backwards to review screens. 
The computer does not provide feedback. 

ASVAB Tests in Study 2 

Study 2 involved use of the same 10 ASVAB tests as described for Study 1, above. 



LAMP Tests in Study 2 

Choice Reaction Time. This is the same test as was used in Study 1, above. 

Physical Identity. This is the same test as was used in Study 1, above. 

Name Identity. This is the same test as was used in Study 1, above. 

General Knowledge Test. Subjects are required to enter the first two letters of the single- 
word answers to 24 general knowledge questions such as "What is the name of the molten rock that 
flows down the sides of an active volcano?" (LAVA). The questions were selected from an earlier 100- 
item form using item-analysis procedures. The test is scored for both percent correct and latency. 

Word Association Test. The subject is asked to associate 9 pairs of words in each of four study 
blocks. The word pairs in a block are presented one at-a-time at a rate of 4 seconds per pair. Each 
study block is followed by 9-alternative multiple-choice items in which the subject is presented with 
one member word of a pair and asked to identify the other member word with which it was associated. 
The distractors for each item arc selected from among the 18 words used in the immediate block. The 
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subject is provided with latency and accuracy feedback for each item, and summary accuracy and 
latency data at the end of each block. 



Figure Recognition Test, In this test, figures are drawn by connecting lines between points in 
an invisible 9 by 9 dot matrix. Of the over one-million potential figures, 160 were selected which were 
judged to be at least moderately encodable. In each of four blocks, 20 of the figures were displayed in 
a 5 by 4 matrix for a 75-second study time. The subjects are asked to study each figure so that they will 
recognize it if they see it again. Subjects are encouraged to give meaning to figures by associating 
them with known objects or scenes. At the end of the 75-second study period, 40 figures are presented 
one-at-a-time. If a figure is judged to be one of the 20-figure study set, the subject is asked to press the 
'U key. Otherwise, the subject is asked to press the T>' key. Accuracy and latency feedback is 
provided after each response, and summary feedback is provided at the end of each block. This test is 
scored for percent correct only. 

Mental Character Generation. In this test, the subjects are asked to commit to memory the 
numbers from 1 to 9 in the relative locations shown below: 

1 4 7 

2 5 8 

3 6 9 

For a particular item, subjects are asked to mentally connect numbers with lines in accordance 
with instructions displayed on successive screens. While processing a given screen, the subjects must 
retain a mental image of the figure drawn by previous instructions, and add to this figure the line called 
for by the present instruction. If the instructions are properly executed, at the end of the last 
instruction in a set, the mental image of a letter or number will have been generated. The subjects are 
asked to record this letter or number by pressing the corresponding key. For example, here are the 
instructions for one of the items: 

screen 1: connect 3 to 1 
screen 2: connect 7 to 6 
screen 3: connect 1 to 6 
screen 4: ? 

The correct answer to the above item is the letter *V\ There are only 13 items in this test. 
Two of them involve connecting 2 lines, two, 3 lines; four, 4 lines; three, 5 lines; and two, 6 lines. The 
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subjects are permitted to move at their own pace, and feedback is provided on accuracy. This test was 
designed to be a measure of working memory involving spatial materials. It requires storing a mental 
image while processing additional visual information. 



Tests used in Study 3 

Study 3 did not involve administration of any tests from the LAMP program. It involved only 
the administration of three alternate forms of tests from the ASVAB (Arithmetic Reasoning, 
Mathematics Knowledge, and Mechanical Comprehension). These tests were selected as the three 
ASVAB tests that were most predictive of learning in the Logic Gates Course. The purpose of Study 3 
was to evaluate the possible influence of concurrency on computed validity coefficients. 
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APPENDIX B 



FACTOR ANALYSIS OF STUDY 1 AND STUDY 2 TEST SCORES 



Study 1 Factor Matrix 

Table Bl Results from Factor Analysis of Study 1 Predictor Tests 

TEST PROCSPD WKMEM PROCACC VERBAL TECHKNOW NUM/PS 



NIL 


.89 


-.04 


.05 


-.01 




- 17 
-.1 / 


PIL 


.81 


-.07 


.17 


.04 


.04 


-.12 


MIL 


.69 


-.10 


-.03 


-.15 


-.02 


-.08 


CRL 


.50 


-.05 


.08 


.01 


-.03 


-.02 


NAA 


-.09 


.74 


.21 


.11 


.08 


.16 


DSA 


-.20 


.58 


.19 


-.09 


.07 


.03 


MK 


.03 


.58 


-.02 


30 


.14 


.36 


AR 


-.02 


.55 


.07 


.20 


.27 


.29 


NRA 


-.15 


.54 


30 


.21 


.08 


.07 


ABA 


.02 


.47 


.30 


34 


.04 


-.08 


PIA 


.15 


.01 


.77 


-.03 


-.00 


.07 


NIA 


-.02 


.05 


.73 


.04 


.03 


.10 


MIA 


.06 


.20 


.68 


.15 


-.03 


-.04 


CRA 


.07 


.13 


.43 


-.04 


.11 


.05 


NFA 


,03 


.22 


.33 


-.03 


.00 


-.01 


ABL 


.23 


.21 


.32 


-.13 


.03 


-.21 


WK 


-.06 


.24 


-.04 


.76 


.15 


-.13 


GS 


.04 


.11 


-.13 


.63 


.37 


-.11 


PC 


-.07 


.04 


.08 


.61 


.15 


.11 


EI 


.05 


.14 


.04 


.31 


.71 


-.07 


AS 


.01 


.02 


.09 


.09 


.69 


-.17 


MC 


-.02 


.32 


.03 


.25 


.60 


-.08 


NO 


-.09 


.17 


.07 


-.06 


-.18 


.67 


NFL 


JO 


-.08 


.15 


.06 


.10 


-.56 


CS 


-.20 


.13 


.14 


-.03 


-.11 


.50 



Notes. See Figure 1 for test titles. Loadings are from Varimax rotations of Principal Axis Factors. 
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Study 1 Factor Descriptions 



FACTOR I- PR OCESSING SPEED 

This factor was defined solely by latency scores from LAMP tests designed to measure 
processing speed, including the Name Identity, Physical Identity, Meaning Identity, Choice Reaction, 
and Number Fact Recall tests. No test from the ASVAB had significant loadings on this factor. 

FACTOR II- WORKING MEMORY 

All of the LAMP tests selected to be measures of working memory had significant loadings on 
this factor. The test with the highest loading was the Number Assignment test (sometimes called the 
ABC Recall Test), which has been the primary definer of working memory in several previous studies 
(Christal. 1988; Kyllonen & Christal, 1989, 1990). The three ASVAB tests with significant loadings on 
this factor were Mathematics Knowledge, Arithmetic Reasoning, and Mechanical Comprehension, all 
of which have had significant loadings on the working memory factor in previous analyses (Christal, 
1988). In the Study 1 factor analysis, it so happened than many of the tests defining the working 
memory factor were based on processing numerical content. However, the fundamental nature of the 
factor is believed to be the requirement for information processing in the presence of a memory load, 
and unpublished LAMP studies suggest that this factor may not be heavily influenced by the type of 
content processed. All of the tests in Study 1 required the subjects to store intermediate results while 
completing the processing demanded for problem solution. 

FACTOR III - PROCESSING ACCURACY 

This factor was defined by accuracy sccres from simple LAMP processing speed measures. 
Most subjects made near-perfect scores on such tests. For example the tests with the top four loadings 
on this factor were Physical Identity, Name Identity, Meaning Identity, and Choice Reaction, for which 
the mean percent accuracy scores were 95.5, 95.4, 93.2, and 98.5, respectively. Although few errors 
were made on these simple processing tasks, the subjects who made the most errors on one of these 
tests tended to be those who also made the most errors on the others. 
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FACTOR IV - VERBAL 



This is the verbal factor which is routinely identified in analyses of ASVAB tests. In the 
present study it was defined by the Word Knowledge, Paragraph Comprehension, and General Science 
tests. None of the LAMP tests were designed to measure verbal knowledge, and none had large 
loadings on this factor. 



FACTOR V - TECHNICAL KNOWLEDGE 

This is the technical knowledge factor which is routinely found in factor analyses of the 
ASVAB tests. It was defined by Electrical Information, Auto and Shop Information, Mechanical 
Comprehension, and General Science. The only LAMP test that had a significant loading on this 
factor was Grammatical Reasoning, which had a loading of only 34. 



FACTOR VI - NUMERICAL FACILITY /PERCEPTUAL SPEED 

Factor analyses of tests from the ASVAB normally yield a doublet factor defined by Numerical 
Operations and Coding Speed. This is often labeled as being a perceptual speed factor. In the present 
analysis, the LAMP Number Fact Recall test joined the two ASVAB tests in defining the factor, and 
had the second highest loading on it. Since two of the three tests defining the factor involve simple 
numerical computations under speeded conditions, it was labeled in this study as being a "Numerical 
Facility/Perceptual Speed" factor. 
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Table BZ Results from Factor Analysis of Study 2 Predictor Tests 



TEST 


VERBAL 


WKMEM 


PROCSPD 


PROCACC 


TECHKNOW 


NUM/PS 


WK 


.73 


.09 


-.04 


.01 


.09 


-.07 


GKA 


.68 


.21 


-.06 


-.02 


.20 


-.11 


GS 


.64 


.21 


-.06 


-.01 


38 


-.00 


GKL 


-.59 


-.10 


.21 


.11 


-.08 


-.35 


PC 


.JJ 


05 


- 06 


04 


.05 


.13 


NAA 


.07 


.71 


-.04 


.22 


.00 


.17 


MK 


.19 


.59 


-.01 


-.03 


.02 


.43 


AD 


11 

. JLJL 




06 


-04 


.21 


.47 


MCA 


.14 


.51 


-.10 


.12 


.13 


.02 


FRA 


.06 


.50 


-.24 


.23 


.09 


-.18 


WAA 


30 


.46 


-.23 


.20 


.06 


-.11 


ABA 


.36 


.43 


-.00 


.25 


.08 


.12 


PIL 


-.06 


-.07 


.84 


.18 


-.03 


-.19 


NIL 


-.16 


-.03 


.84 


.13 


.09 


-.22 


CRL 


-.08 


-.17 


.51 


.17 


-.01 


-.05 


PIA 


-.00 


.07 


.12 


.76 


-.01 


.14 


NIA 


.07 


.10 


.08 


.66 


-.00 


.08 


CRA 


-.03 


.18 


.11 


.46 


.12 


-.00 


ABL 


-.05 


.26 


.20 


.42 


-.08 


-.19 


AS 


.11 


-.05 


.06 


-.01 


.78 


-.12 


MC 


.19 


.32 


-.02 


-.02 


.63 


-.03 


EI 


.29 


.10 


-.01 


.13 


.63 


-.06 


NO 


.02 


.07 


-.18 


.07 


-.10 


.63 


CS 


.00 


.04 


-.19 


.08 


-.15 


.60 


Notes. See Figure 1 for test titles. 



Factor Descriptions 

The factor descriptions for the Study 2 analysis are identical to those for Study 1, with the 
following exceptions: 1) the Working Memory factor is now defined by spatial and verbal tests, in 
addition to quantitative tests; 2) the Processing Speed factor is a little weaker because the Meaning 
Identity test was no longer in the battery; and 3) the Numerical Facility/Perceptual Speed factor 
contains less numerical variance because the Number-Fact Retrieval test had been dropped. 
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APPENDIX C 

VALIDITIES OF ASVAB AND LAMP TESTS FOR STUDY 1 AND STUDY 2 ACCURACY AND 

LATENCY CRITERIA 



Table CI. Validities of ASVAB and LAMP tests for Study 1 Accuracy Criteria 

BLOCK AVERAGES 
TEST Avl Ay2 AVG 



ASVAB 

GS .06 .15 .13 

AR 38 33 .40 

WK .13 .22 .20 

PC .09 .20 .16 

NO .12 .03 .08 

CS .12 -06 .09 

AS .05 .12 .10 

MK 34 .36 .40 

MC .20 31 .29 

EI .16 .28 .26 

LAMP 

CRA .25 .24 .27 

NIA 26 .25 .29 

PIA .28 .21 .27 

DSA 31 .28 32 

NFA .18 .16 .19 

NRA .41 .43 .47 

MIA 33 31 35 

ABA .40 .40 .44 

NAA .47 .39 .48 

CRL .06 .02 .05 

PIL .11 -08 .11 

NFL .03 .09 .07 

MIL .10 .02 .07 

ABL .28 .34 .34 

NIL .09 ;06 -09 

Notes. Block codes are defined in Figure 2. Test abbreviations are defined in Figure 1. 
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Table C2. Validities of ASVAB and LAMP tests for Study 1 Latency Criteria 

BLOCK AVERAGES 
TEST Avl Ay2 AVG 

ASVAB 

GS -.04 .04 -.01 

AR -.13 .02 -.06 

WK -.06 .01 -.03 

PC -.04 -.01 -.03 

NO -.10 -.12 -.13 

CS -.10 -.08 -.10 

AS -.03 .05 .01 

MK -.10 .00 -.06 

MC -.01 .01 .03 

EI .00 .07 .05 

LAMP 

CRA .05 -.01 .05 

NLA .03 .06 .07 

PIA .04 .10 .08 

DSA -.12 -.04 -.07 

NFA .00 .07 .04 

NRA -.14 -.01 -.07 

MIA .05 .17 .14 

ABA -.05 .12 .03 

NAA -.17 -.04 -.11 

CRL .16 .19 .14 

PIL .33 .29 .32 

NFL .28 .34 .31 

MIL .19 .25 .21 

ABL .23 .40 .35 

NIL 35 34 36 

Notes. Block codes are defined in Figure 2. Test abbreviations are defined in Figure 1. 
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Table C3. Validities of ASVAB and LAMP Tests for Study 2 Accuracy Criteria 


TEST 


Avl 


2-1 


BLOCK AVERAGES 
Av3 


Av4 


AVG 


ASVAB 












GS 


.13 


21 


.20 


.22 


.21 


AR 


.33 


.35 


.33 


.26 


34 


WK 


.11 


.15 


.19 


.16 


.17 


PC 


.07 


.11 


.11 


.13 


.12 


NO 


.14 


.14 


.14 


.12 


.15 


CS 


.13 


.10 


.07 


.08 


.11 


AS 


-.03 


.13 


.13 


.08 


.07 


MK 


.40 


33 


.32 


30 


.38 


MC 


.21 


.27 


31 


.26 


.29 


EI 


.08 


.20 


.20 


.19 


.18 


LAMP 












GKA 


.14 


.15 


.14 


.16 


.17 


FRA 


.34 


.26 


.28 


34 


.36 


MCA 


37 


.29 


.34 


39 


.40 


NIA 


.20 


.25 


.22 


.20 


.23 


PIA 


.25 


.18 


.20 


.19 


.24 


CRA 


.30 


.17 


.20 


.26 


.28 


ABA 


.39 


.36 


.38 


.41 


.44 


WAA 


35 


.31 


.33 


.36 


.39 


NAA 


.44 


.36 


.36 


.43 


.46 


WKL 


-.10 


-.16 


-.16 


-.15 


-.15 


NIL 


-.11 


-.07 


-.07 


-.11 


-.11 


PIL 


-.09 


-.04 


-.05 


-.08 


-.08 


ABL 


.24 


.21 


.24 


.33 


.30 


CRL 


-.12 


-.17 


-.17 


-.18 


-.17 



Notes. Block codes defined in Figure 2. Test abbreviations defined in Figure 1. The values under 2-1 
are for a single block of simple gates with negations- 
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Table C4. Validities of ASVAB and LAMP Tests for Study 2 Latency Criteria 



BLOCK AVERAGES 
TEST Avl 2-1 Ar3 Av4 AVG 



ASVAB 



GS 


-.19 


-.15 


1 O 

-.18 


-.07 


1 C\ 

-.IV 


AR 


-.23 


-.17 


-.14 


-.09 


-.18 


WK 


-.09 


-.01 


-.04 


-.02 


-.06 


PC 


-.06 


-.06 


-.09 


-.01 


-.07 


NO 


-.14 


-.09 


-.10 


-.13 


-.12 


cs 


-.08 


-.06 


-.13 


-.14 


-.11 


AS 


-.02 


-.09 


-.01 


-.00 


-.02 


MK 


-.22 


-.14 


-.14 


-.06 


-.17 


MC 


-.14 


-.11 


-.05 


.01 


-.08 


EI 


-.09 


-.04 


-.05 


.05 


-.05 


LAMP 












GKA 


-.22 


-.17 


-.19 


-.10 


-.21 


FRA 


-.22 


-.13 


-.13 


.01 


-.15 


MCA 


-.23 


-.14 


-.11 


.04 


-.13 


NIA 


-.09 


.02 


.04 


.01 


-.02 


PIA 


-.02 


.05 


.08 


.03 


.04 


CRA 


-.03 


.02 


.06 


.18 


.07 


ABA 


-.17 


-.07 


-.03 


.07 


-.07 


WAA 


-.27 


-.09 


-.11 


.07 


-.14 


NAA 


-.27 


-.16 


-.13 


.06 


-.14 


GKL 


.26 


.24 


.25 


.18 


.27 


NIL 


.38 


.25 


.35 


.22 


.37 


PIL 


.37 


.28 


.36 


.25 


.38 


ABL 


.10 


.22 


.26 


.41 


.29 


CRL 


.33 


.26 


.23 


.15 


.28 



Notes. Block codes defined in Figure 2. Test abbreviations defined in Figure 1. The values under 2-1 
are for a single block of simple gates with negations. 
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5s 



> 

>- 
o 

oo 

LU 
CD 



CO 
CO 
UJ 

z 

CO 
CD 

< 

O 



< 

s 

Q. 



ERIC 



